Page MenuHomePhabricator
Create Task
Maniphest T259637

Gather information about the volume of queries on WCQS
Closed, ResolvedPublic5 Estimated Story Points
Actions

Description

As a Wikimedia staff member, I want to know more information about the volume of queries on WCQS so that I can use the information to make decisions about the future direction of WCQS.

Acceptance Criteria:

  • Provide data about the volume of queries on WCQS
  • Format of the data should be CSV
  • Data should be updated weekly

Details

SubjectRepoBranchLines +/-
Enable file query events for WCQSoperations/puppetproduction+19 -1
Allow event sender if header overwritewikidata/query/rdfmaster+2 -1
Summarizng tool for file query eventswikidata/query/rdfmaster+255 -0
Add file based query event senderwikidata/query/rdfmaster+212 -21
Customize query in gerrit

Event Timeline

CBogen created this task.Aug 4 2020, 4:18 PM
Restricted Application added a project: Wikidata. · View Herald TranscriptAug 4 2020, 4:18 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
CBogen added a comment.Aug 4 2020, 4:19 PM

@dcausse what format is best for this, and is it something that can be run/updated weekly showing change over time? Can it be in Superset, for example?

dcausse added a comment.Aug 6 2020, 3:27 PM

I think first we could generate a CSV file accessible from wcqs-beta-01. I'm not very familiar with dashboard tools like Superset so I don't know how to push the data there from wmcloud.

CBogen added a subscriber: EBernhardson.Aug 6 2020, 3:30 PM

@EBernhardson any advice on whether we can push this data to Superset?

EBernhardson added a comment.Aug 6 2020, 3:53 PM

It's hard to get this into superset because it doesn't live in the production network, the only access it has to get data to superset would be through our public endpoints. Off the top of my head, the only thing i can think of is submitting events to the public beacon endpoint like a browser would, not sure how workable that is.

CBogen added a comment.Aug 7 2020, 2:41 PM

Okay, for now then a CSV would be fine, and we can talk about putting it into superset eventually when WCQS moves to production.

Can we do the CSV weekly? Monthly would also be okay, I think.

Gehel moved this task from Incoming to Current work on the Wikidata-Query-Service board.Aug 10 2020, 1:46 PM
Gehel added a project: Discovery-Search (Current work).
CBogen updated the task description. (Show Details)Aug 10 2020, 3:41 PM
CBogen set the point value for this task to 5.Aug 10 2020, 3:43 PM
CBogen moved this task from Incoming to Ready for Dev -- SWE on the Discovery-Search (Current work) board.
• Zbyszko claimed this task.Aug 11 2020, 7:08 AM
• Zbyszko removed • Zbyszko as the assignee of this task.
• Zbyszko moved this task from Ready for Dev -- SWE to In Progress on the Discovery-Search (Current work) board.
• Zbyszko moved this task from In Progress to Ready for Dev -- SWE on the Discovery-Search (Current work) board.
• Zbyszko subscribed.
• Zbyszko claimed this task.Aug 11 2020, 1:26 PM
• Zbyszko moved this task from Ready for Dev -- SWE to In Progress on the Discovery-Search (Current work) board.
gerritbot added a comment.Aug 13 2020, 7:05 AM

Change 619948 had a related patch set uploaded (by ZPapierski; owner: ZPapierski):
[wikidata/query/rdf@master] Add file based query event sender

https://gerrit.wikimedia.org/r/619948

gerritbot added a project: Patch-For-Review.Aug 13 2020, 7:05 AM
gerritbot added a comment.Aug 18 2020, 2:27 PM

Change 620961 had a related patch set uploaded (by ZPapierski; owner: ZPapierski):
[wikidata/query/rdf@master] Summarizng tool for file query events

https://gerrit.wikimedia.org/r/620961

• Zbyszko moved this task from In Progress to Needs review on the Discovery-Search (Current work) board.Aug 19 2020, 8:48 AM
gerritbot added a comment.Aug 21 2020, 9:36 AM

Change 619948 merged by jenkins-bot:
[wikidata/query/rdf@master] Add file based query event sender

https://gerrit.wikimedia.org/r/619948

gerritbot added a comment.Aug 21 2020, 9:36 AM

Change 620961 merged by jenkins-bot:
[wikidata/query/rdf@master] Summarizng tool for file query events

https://gerrit.wikimedia.org/r/620961

Maintenance_bot removed a project: Patch-For-Review.Aug 21 2020, 10:10 AM
gerritbot added a comment.Aug 21 2020, 11:34 AM

Change 621708 had a related patch set uploaded (by ZPapierski; owner: ZPapierski):
[wikidata/query/rdf@master] Allow event sender if header overwrite

https://gerrit.wikimedia.org/r/621708

gerritbot added a project: Patch-For-Review.Aug 21 2020, 11:34 AM
gerritbot added a comment.Aug 21 2020, 11:43 AM

Change 621710 had a related patch set uploaded (by ZPapierski; owner: ZPapierski):
[operations/puppet@production] Enable file query events for WCQS

https://gerrit.wikimedia.org/r/621710

gerritbot added a comment.Aug 21 2020, 12:20 PM

Change 621708 merged by jenkins-bot:
[wikidata/query/rdf@master] Allow event sender if header overwrite

https://gerrit.wikimedia.org/r/621708

gerritbot added a comment.Aug 21 2020, 7:10 PM

Change 621710 merged by Ryan Kemper:
[operations/puppet@production] Enable file query events for WCQS

https://gerrit.wikimedia.org/r/621710

• Zbyszko moved this task from Needs review to To Be Deployed on the Discovery-Search (Current work) board.Aug 24 2020, 5:41 PM
• Zbyszko moved this task from To Be Deployed to Needs Reporting on the Discovery-Search (Current work) board.Sep 15 2020, 2:06 PM
Gehel closed this task as Resolved.Sep 28 2020, 2:44 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL