Page MenuHomePhabricator

Monitor commons-query.wikidata.org
Open, LowPublic

Description

Per IRC conversation today in #wikimedia-sre , community member @Dominicbm reported commons-query.wikidata.org (the web UI for WCQS) as down. Further troubleshooting by @EBernhardson , @dcausse and @Dzahn revealed an issue with firewall rules . Once the linked changes were reverted, the site returned to service.

However, the site *was* down for ~1 week without the responsible SRE team (Data Platform SRE, which includes myself) noticing. To prevent this from happening in the future, we should add monitoring for commons-query.wikidata.org. Creating this ticket to:

  • Add a blackbox check for commons-query.wikidata.org
  • Add alerts for this site.

Event Timeline

Don't bother, nobody uses it anyway as long as it has authentication. That should be removed, see T297995. Not even clear what the future will be, see T376979

Both T350793#10378435 and today T350793#10394688 happened and caused realtime debugging. This seems to be at odds with the claim that nobody uses it anyways.

See class profile::microsites::monitoring in the puppet repo for examples to copy/paste/adjust to get the Blackbox check.

Both T350793#10378435 and today T350793#10394688 happened and caused realtime debugging. This seems to be at odds with the claim that nobody uses it anyways.

The usage of the WDQS can be seen at https://grafana.wikimedia.org/d/000000489/wikidata-query-service?orgId=1&var-cluster_name=wdqs&var-graph_type=%289102%7C919%5B35%5D%29&refresh=1m&from=now-7d&to=now . It hovers between 5000 and 15000 requests per minute. The Commons query service can be seen at https://grafana.wikimedia.org/d/000000489/wikidata-query-service?orgId=1&var-cluster_name=wcqs&var-graph_type=%289102%7C919%5B35%5D%29&refresh=1m&from=now-7d&to=now and has some spikes of maybe 20 queries per minute and whole periods of no queries at all. The main user seems to have been the monitoring system ( https://grafana.wikimedia.org/d/000000489/wikidata-query-service?orgId=1&var-cluster_name=wcqs&var-graph_type=%289102%7C919%5B35%5D%29&refresh=1m&from=now-90d&to=now&viewPanel=44 ) which disappears 16th October. So any time spend on this is wasted effort.

Gehel triaged this task as Low priority.Jan 8 2025, 8:28 AM
Gehel moved this task from Incoming to Observability on the Data-Platform-SRE board.