Page MenuHomePhabricator

Investigate the number of queries on DCAT endpoint
Open, MediumPublic

Description

As a product manager, I want to know the volume of query traffic that the DCAT endpoint gets, so that I can make a cost/benefit decision on whether to remove this endpoint.

DCAT is a namespace in Blazegraph where we load category trees. We suspect that it is relatively low traffic often enough that the maintenance cost does not make it worth supporting -- removing it will also allow the team to focus on scaling WDQS.

This ticket is to check how many queries we have on this endpoint, and how many queries would be impacted by removing it.

AC:

  • determine average number of queries on DCAT (per day/week/month)

Event Timeline

It is serving the Data Catalog Vocabulary so it's not used by deepcat.
It is just offering sparql endpoint on top of the file https://dumps.wikimedia.org/other/wikibase/wikidatawiki/dcatap.rdf.
I think the main benefit of this dataset is the links to the dumps but I wonder if a sparql endpoint is interesting.
While reviewing older tasks it looks like we:

  • we added the dcatap namespace to production machines (T178978)
  • moved it dcatap.wmflabs.org (T228297)
  • removed production dcatap namespace
  • broke dcatap.wmflabs.org (T289517)
  • currently trying to re-enable dcatap in production

Overall I feel that this sparql endpoint either requires more love or needs to be removed :)