Page MenuHomePhabricator
Create Task
Maniphest T239908

Extract more metrics from blazegraph sparql update response
Closed, ResolvedPublic
Actions

Description

As of today when we run an update to blazegraph we only extract the total number of mutations performed.
But the current update process runs between 5 and 7 update queries in one request. Having numbers about how each query performed might be interesting, esp. to answer some of the question asked in T239687.

The goal is to make org.wikidata.query.rdf.tool.rdf.RdfRepository#syncFromChangesNonMerging reports all the known metrics for all the individual update requests we send in this batch.
Because blazegraph only reports HTML for update response something similar to org.wikidata.query.rdf.tool.rdf.client.UpdateCountResponse will have to be added.

Related Objects

Event Timeline

• dcausse created this task.Dec 5 2019, 1:40 PM
Restricted Application added a project: Wikidata. · View Herald TranscriptDec 5 2019, 1:40 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
• Zbyszko claimed this task.Dec 12 2019, 8:39 AM
• dcausse removed • Zbyszko as the assignee of this task.Dec 12 2019, 8:43 AM
• dcausse assigned this task to • Zbyszko.
• dcausse triaged this task as Medium priority.
• dcausse updated the task description. (Show Details)
• dcausse added a project: Discovery-Search (Current work).
• dcausse added a subscriber: • Zbyszko.
Addshore moved this task from incoming to in progress on the Wikidata board.Dec 12 2019, 10:42 AM
gerritbot added a comment.Dec 13 2019, 3:40 PM

Change 557048 had a related patch set uploaded (by ZPapierski; owner: ZPapierski):
[wikidata/query/rdf@master] ResponseHandler that extract every single stat available from Blazegraph response

https://gerrit.wikimedia.org/r/557048

gerritbot added a project: Patch-For-Review.Dec 13 2019, 3:40 PM
gerritbot added a comment.Jan 7 2020, 10:08 AM

Change 562478 had a related patch set uploaded (by ZPapierski; owner: ZPapierski):
[wikidata/query/rdf@master] Smoke test to ensure query modification safety

https://gerrit.wikimedia.org/r/562478

gerritbot added a comment.Jan 7 2020, 4:49 PM

Change 562478 merged by jenkins-bot:
[wikidata/query/rdf@master] Smoke test to ensure query modification safety

https://gerrit.wikimedia.org/r/562478

gerritbot added a comment.Jan 9 2020, 9:00 AM

Change 557048 merged by jenkins-bot:
[wikidata/query/rdf@master] ResponseHandler that extract every single stat available from Blazegraph response

https://gerrit.wikimedia.org/r/557048

Maintenance_bot removed a project: Patch-For-Review.Jan 9 2020, 9:10 AM
• Zbyszko moved this task from Incoming to To Be Deployed on the Discovery-Search (Current work) board.Jan 9 2020, 10:23 AM
Daniel_Mietchen subscribed.Feb 8 2020, 1:24 PM
• dcausse moved this task from To Be Deployed to Needs Reporting on the Discovery-Search (Current work) board.Feb 19 2020, 10:16 AM

Dashboard created here: https://grafana.wikimedia.org/d/dSksY08Zk/wikidata-query-service-updater?orgId=1

Gehel closed this task as Resolved.Feb 26 2020, 4:13 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits