Page MenuHomePhabricator

smartmeta.wikibase.cloud and others are missing some updates to the queryservice
Closed, ResolvedPublic

Description

Via Email from user

Some examples:

Query SPARQL with not results
https://tinyurl.com/2geu8hre
https://tinyurl.com/2o8rt7al

Event Timeline

Looking through the logs I seem lots of:

java.lang.IllegalStateException: RDF store reports the last update time is before the minimum safe poll time.  You will have to reload from scratch or you might have missing data.
	at org.wikidata.query.rdf.tool.change.ChangeSourceContext.getStartTime(ChangeSourceContext.java:100)
	at org.wikidata.query.rdf.tool.WbStackUpdate.initialize(WbStackUpdate.java:305)
	at org.wikidata.query.rdf.tool.WbStackUpdate.runUpdaterWithArgs(WbStackUpdate.java:229)
	at org.wikidata.query.rdf.tool.WbStackUpdate.updateBatch(WbStackUpdate.java:207)
	at org.wikidata.query.rdf.tool.WbStackUpdate.getAndProcessBatchesFromApi(WbStackUpdate.java:163)
	at org.wikidata.query.rdf.tool.WbStackUpdate.main(WbStackUpdate.java:133)

This looks very much like a re-occurance of T309692. We should investigate how this happened? Did we fail to remove the date triples for people migrated in later batches?

Did we fail to remove the date triples for people migrated in later batches?

Yes we did. I chatted to @Evelien_WMDE and we're going to go ahead and remove all these erroneous date triples (with the code documented in T309692) after doing T312623 (so this user's wikis aren't also caught by it after 30days)

Queryservice Problem should now be resolved for new edits.

Determined the list of namespaces for wikis made after the 2nd of june and ran the the script:

$wikis=Wiki::whereDate('created_at', '>=', '2022-06-02')->get();
$wikis->map(function ($wiki) {
return $wiki->wikiQueryserviceNamespace->namespace;
})

Passed to: bin/queryservice-delete-dump-timestamp-in-namespaces

We should review this by confirming that we no longer see any instances of these errors in the updater logs

Tarrow renamed this task from smartmeta.wikibase.cloud is missing some updates to the queryservice and blazegraph to smartmeta.wikibase.cloud and others are missing some updates to the queryservice.Jul 11 2022, 6:02 PM
Tarrow updated the task description. (Show Details)

It's not absolute solid proof but it looks promising:

image.png (119×1 px, 9 KB)

I searched for this error java.lang.IllegalStateException: in the log and its earliest occurrence is 2022-07-11 which the date it was resolved

@Evelien_WMDE: The project tag got archived and this open task has no other active project tags. Could you please either add an active project tag so this task can be found, or update the task status? Thanks a lot!