In the Commons query service, the results for a query are varying wildly depending on which server handles the request, with queries sometimes returning a lot less data than expected, or no data at all.
Counting the total number of triples (query) and looking at the x-served-by header, it seems that wcqs1002 and wcqs1003 have far less data than wcqs1001: wcqs1001 says 5.7 billion, while wcqs1002 and wcqs1003 say 53 million.
Thanks for reporting this. I consider WCQS pretty much broken in this state, since its results are unreliable and not even showing any sort of error message making this clear. You can do things like changing the whitespace to trigger a refresh of the query, but it's not obvious when you've gotten the "real" answer except when it feels right.
Thanks for bringing this to our attention, and sorry for the inconvenience. I have depooled the eqiad datacenter while we work to address the issue. Please test our your queries and let us know if you're still getting inconsistent results.
I've repooled wcqs1001 and wcqs1002 after verifying they have the correct amount of triples. We've left wcqs1003 depooled as we troubleshoot further. Let us know if you notice any other issues.
Mentioned in SAL (#wikimedia-operations) [2023-08-24T17:23:36Z] <ryankemper> [WCQS] T344882 ryankemper@wcqs1003:~$ sudo depool
Hello,
I've fixed wdqs1003 as well and it appears we have the same amount of triples now. As such, I'm resolving this ticket. But...please let us know if you notice anything else amiss.