Ideally Cloud-Services will liaise with @Magnus either to bottom out & solve the issue; or else to provide an explanation for the 502 issue which we can use to console ourselves each time it happens.

OK, so what appears to happen is that SQL queries timeout and take PetScan with them. Note:

• I wrote some code that re-arranges certain large queries into smaller ones, which cuts down on the timeouts; that code has been live for weeks
• That works fine on the dev machine but not reliably on the production machine
• The dev machine has less resources than production, but is otherwise identical (OS etc)

As this does not fail reproducibly, it's either some odd bug in my code, or some situation on the DB replicas.

Also, I just clicked on the two examples. They took 133 and 173 seconds, and returned 50 and 1 results, respectively. No 502s, though I have seen those occasionally.

Thanks Magnus. I run those two daily; get perhaps 20% 502s, without an obvious pattern. There are others (e.g. listed on https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Women_in_Red/Metrics/Wikidata ) which more reliably 502

I think it's for you to decide whether to hold this ticket open or close it.

502 examples in the last couple of minutes:

http://petscan.wmflabs.org/?language=en&project=wikipedia&depth=4&categories=people%20by%20occupation&ns%5B0%5D=1&sparql=select%20%3Fitem%20where%20%7B%3Fitem%20wdt%3AP31%20wd%3AQ5%7D&wikidata_item=with&wikidata_prop_item_use=P21&wpiu=none&interface_language=en&

http://petscan.wmflabs.org/?psid=6220454&al_commands=P21%3AQ6581097%0AP31%3AQ5

Testing on dev now. Looks like frequent Lost connection to MySQL server during query errors from the DB replicas.

Sorry to burden you with this. As a user 502s are v.frustrating, but I can also imagine it's v.frustrating for you to be asked to debug when you could be making snowmen instead.

Try it now...

Mixed results. The following three worked like a charm:

*Articles with no wikidata item
:*to 5 levels of depth - [http://petscan.wmflabs.org/?psid=5984708&al_commands=P31%3AQ5%0AP21%3AQ6581072 auto-run]
:*to 6 levels of depth - [http://petscan.wmflabs.org/?psid=7075418&al_commands=P31%3AQ5%0AP21%3AQ6581072 auto-run]
:*to 7 levels of depth - [http://petscan.wmflabs.org/?psid=7075446&al_commands=P31%3AQ5%0AP21%3AQ6581072 auto-run]

But the next three all 502d

*Articles, with wikidata items specifying 'human' but with no gender
:*to 5 levels depth - [http://petscan.wmflabs.org/?psid=5823558&al_commands=P21%3AQ6581072 auto-run]
:*to 6 levels depth - [http://petscan.wmflabs.org/?psid=7075471&al_commands=P21%3AQ6581072 auto-run]
:*to 7 levels depth - [http://petscan.wmflabs.org/?psid=7075514&al_commands=P21%3AQ6581072 auto-run]

And to prove the curate's egg, http://petscan.wmflabs.org/?psid=7075446&al_commands=P31%3AQ5%0AP21%3AQ6581072 when run again, 502d.

There are some PetScan outages, reason unknown so far, but outside those, the above queries all work fine.

I did/do not see any successful Petscan query for days. Always getting a: 504 Gateway Time-out.

Same here. 504 Gateway Time-out on http://petscan.wmflabs.org/.

@Aklapper actually the repo is https://github.com/magnusmanske/petscan_rs (the bitbucket one is the old C++ version).

@Magnus: Ah, thanks! Could be nice to add such documentation to https://wikitech.wikimedia.org/wiki/Nova_Resource:Petscan ?

For smaller queries, I think http://petscan-dev.wmflabs.org/. I don't know about it in detail.

It's my dev server, should work for any query size. Feel free to use when the main site is down.