Page MenuHomePhabricator

Wikidata grafana dashboard for number of instances of various things is broken
Closed, ResolvedPublic5 Story Points

Description

PHP Fatal error:  Uncaught RuntimeException: The SPARQL request failed! in /srv/analytics-wmde/graphite/src/scripts/src/wikidata/sparql/instanceof.php:78
Stack trace:
#0 /srv/analytics-wmde/graphite/src/scripts/src/wikidata/sparql/instanceof.php(59): WikidataInstanceOf->doSparqlQuery('PREFIX wd: <htt...')
#1 /srv/analytics-wmde/graphite/src/scripts/src/wikidata/sparql/instanceof.php(40): WikidataInstanceOf->getResult('Q11266439')
#2 /srv/analytics-wmde/graphite/src/scripts/src/wikidata/sparql/instanceof.php(14): WikidataInstanceOf->execute()
#3 {main}
  thrown in /srv/analytics-wmde/graphite/src/scripts/src/wikidata/sparql/instanceof.php on line 78

It looks like this is due to the hardcoded server wdqs1003 no longer existing.
https://github.com/wikimedia/analytics-wmde-scripts/blob/master/src/wikidata/sparql/instanceof.php#L74
perhaps wdqs1005 should be used?
See T176875: Allow access to wdqs.svc.eqiad.wmnet on port 8888 for why we can not just use the service alias.

Example broken dashboards

https://grafana.wikimedia.org/d/000000175/wikidata-datamodel-statements?refresh=30m&orgId=1

Event Timeline

Addshore created this task.Mar 19 2019, 5:46 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 19 2019, 5:46 PM

It could be that the sparql query now takes too long and we perhaps need to think about a better way to get this data?

Hadoop perhaps? :)

Looks like this is because the host needs to be hard coded in the script.

https://github.com/wikimedia/analytics-wmde-scripts/blob/master/src/wikidata/sparql/instanceof.php#L74

Because T176875: Allow access to wdqs.svc.eqiad.wmnet on port 8888 can not be done.

So the host needs to be updated!
Perhaps to wdqs1005?

Addshore triaged this task as Low priority.Jun 22 2019, 10:12 PM
Addshore updated the task description. (Show Details)
Addshore moved this task from incoming to ready to go on the Wikidata board.
Addshore moved this task from Incoming to Ready to estimate on the Wikidata-Campsite board.
Addshore updated the task description. (Show Details)
alaa_wmde updated the task description. (Show Details)Jul 2 2019, 12:24 PM
alaa_wmde updated the task description. (Show Details)
alaa_wmde set the point value for this task to 5.Jul 2 2019, 12:27 PM

Change 520901 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/puppet@production] statistics: Add wdqs host to wmde statistcs configuration

https://gerrit.wikimedia.org/r/520901

Change 520903 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[analytics/wmde/scripts@master] Use config for wdqs host name

https://gerrit.wikimedia.org/r/520903

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptJul 5 2019, 3:18 PM

My patch fixes it, tested it on production.

Change 520901 merged by Filippo Giunchedi:
[operations/puppet@production] statistics: Add wdqs host to wmde statistcs configuration

https://gerrit.wikimedia.org/r/520901

Addshore moved this task from ready to go to in progress on the Wikidata board.Jul 9 2019, 4:41 PM

Change 520903 merged by jenkins-bot:
[analytics/wmde/scripts@master] Use config for wdqs host name

https://gerrit.wikimedia.org/r/520903

Change 523131 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[analytics/wmde/scripts@production] Use config for wdqs host name

https://gerrit.wikimedia.org/r/523131

Change 523131 merged by jenkins-bot:
[analytics/wmde/scripts@production] Use config for wdqs host name

https://gerrit.wikimedia.org/r/523131