Page MenuHomePhabricator

Query repeatedly gives "Unknown error"
Closed, ResolvedPublic

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Just debugged:
Status: 200
Exception: JSON parse error

Change 316579 had a related patch set uploaded (by Jonas Kress (WMDE)):
Show JSON parse error

https://gerrit.wikimedia.org/r/316579

@Nikki As a side note - please avoid running queries like "give me ten million entries", unless you really really really need it. It most probably would not be stable, and it creates load on the service that makes harder for others to use it - we're still running on only two machines. And you browser may not be happy loading several hundred megs of JSON into memory either.

This particular query produces hundreds of megabytes of data. I don't really consider doing such thing a recommended use case for the service - if you really need data to the tune of several hundred megs, processing dump is the best venue for now. When we get LDF implementation, there would be other ways too.

That property has only been used about 4000 times, I didn't expect that to translate into ridiculous amounts of data. I don't want or need ten million entries, this isn't even the query I actually wanted results from.

What I was trying to do was get statements with no references for a couple of thousand items. It sounded like it would be pretty straightforward, select some items, select their statements, filter out any with a reference... but apparently not, since my queries kept timing out. I simplified the query and simplified it some more, tried different properties, etc, trying to work out what was causing it to time out. Then suddenly I got to this query and it just said "Unknown error". Up until that point, the queries (excluding the more complicated ones which were timing out) had been returning results within a few seconds and my browser had been having no problems displaying the results so it didn't seem problematic.

I don't see any indication of how large (in kB/MB) a particular response is. If you want people to avoid queries which return a large amount of data, it would help to actually show how big the response is and whether it's considered excessive. If someone does accidentally produce a query which returns too much data, it would better (and more effective) to show an error message saying it produced too much data rather than "Unknown error".

As for what I was originally trying to do, this isn't my original query, but it's a simple demonstration of the problem I was having:
This query returns results in a few seconds but returns lots of stuff I don't want. This query returns results in a fraction of a second. Put them together like this and it times out. As I wrote in T116298 about a completely different query which behaves in the same way, this makes no sense to me.

(WikidataFacts did manage to come up with a version of that query which works - here - but why that works while the original times out is a mystery to me)

@Nikki something like ?item ?prop ?statement is almost always a bad idea, unless you put filters on ?prop and ?statement later. Original one didn't, though the new one does.

Combining statements is not always obvious, and sometimes query optimizer takes wrong turn. We're working on how to make it smarter. However, when you're trying to develop a new query, putting LIMIT on it is not a bad idea, until you're sure it's not going to run away.

If you want people to avoid queries which return a large amount of data, it would help to actually show how big the response is and whether it's considered excessive.

Good idea. We do show row counts but not response size. Maybe we can show both.

it would better (and more effective) to show an error message saying it produced too much data rather than "Unknown error".

These are different issues. "Unknown error" will be handled by GUI, my recommendation is more general regardless of specific errors - please use LIMIT and good judgement :)

Change 316579 merged by jenkins-bot:
Show JSON parse error

https://gerrit.wikimedia.org/r/316579

Jonas moved this task from Proposed to Done on the Wikidata-Sprint-2016-10-12 board.
thiemowmde moved this task from incoming to in progress on the Wikidata board.
thiemowmde removed a project: Patch-For-Review.