Change Details

I'm working on mySociety's Democratic Commons project and the EveryPolitician WikiProject, and we're seeing inconsistencies in the data we get back from the Wikidata Query Service. We're running queries and storing the results in GitHub repositories. To illustrate the problem, here's a commit in which we refresh the data we hold from Wikidata: https://github.com/everypolitician/proto-commons-canada/commit/15c0ae530ede1df864e551659322962672cfe4c6 The data maintained in Wikidata will have barely changed, yet we're seeing large swathes of results coming and going, depending on which WDQS node each query happens to get routed to. This is causing huge trouble for us, as we can't reliably get stable and consistent data out of Wikidata. Would it be possible to have the nodes' data reloaded from scratch to ensure that they're consistent? Would you also be able to investigate the cause of the inconsistency? My colleague, @mhl20 made a previous report about this in T199916 (which I then reopened), and I think the solution was to refresh the data for the entities mentioned, but I think the problem is much wider than that.

I'm working on mySociety's Democratic Commons project and the EveryPolitician WikiProject, and we're seeing inconsistencies in the data we get back from the Wikidata Query Service. We're running queries and storing the results in GitHub repositories. To illustrate the problem, here's a commit in which we refresh the data we hold from Wikidata: https://github.com/everypolitician/proto-commons-canada/commit/15c0ae530ede1df864e551659322962672cfe4c6 The data maintained in Wikidata will have barely changed, yet we're seeing large swathes of results coming and going, depending on which WDQS node each query happens to get routed to. By looking at the `X-Served-By` header on results, I've previously noticed that wdqs1003 had missing data (see T199916#4555158; now fixed), but I haven't yet checked whether other nodes are similarly missing data. This is causing huge trouble for us, as we can't reliably get stable and consistent data out of Wikidata. Would it be possible to have the nodes' data reloaded from scratch to ensure that they're consistent? Would you also be able to investigate the cause of the inconsistency? My colleague, @mhl20 made a previous report about this in T199916 (which I then reopened), and I think the solution was to refresh the data for the entities mentioned, but I think the problem is much wider than that.

I'm working on mySociety's Democratic Commons project and the EveryPolitician WikiProject, and we're seeing inconsistencies in the data we get back from the Wikidata Query Service. We're running queries and storing the results in GitHub repositories. To illustrate the problem, here's a commit in which we refresh the data we hold from Wikidata: https://github.com/everypolitician/proto-commons-canada/commit/15c0ae530ede1df864e551659322962672cfe4c6 The data maintained in Wikidata will have barely changed, yet we're seeing large swathes of results coming and going, depending on which WDQS node each query happens to get routed to. By looking at the `X-Served-By` header on results, I've previously noticed that wdqs1003 had missing data (see T199916#4555158; now fixed), but I haven't yet checked whether other nodes are similarly missing data. This is causing huge trouble for us, as we can't reliably get stable and consistent data out of Wikidata. Would it be possible to have the nodes' data reloaded from scratch to ensure that they're consistent? Would you also be able to investigate the cause of the inconsistency? My colleague, @mhl20 made a previous report about this in T199916 (which I then reopened), and I think the solution was to refresh the data for the entities mentioned, but I think the problem is much wider than that.