Page MenuHomePhabricator
Create Task
Maniphest T326637

Generate metrics for page saving and parsing times, to monitor impact of map external data expansion
Open, Needs TriagePublic
Actions

Description

  • Monitor slow SPARQL queries
    • Log whenever external data fetch takes more than 1 second.
    • Record event with details about:
      • wiki
      • page and revision being parsed
      • ExternalData service: geoshape, geomask, geoline, geopoint
      • external data URL (can be truncated, but include ids and some query)
      • elapsed wall time for this request
      • was the request successful?
      • size of retrieved geojson
    • Average and p95 time to save a page: https://grafana.wikimedia.org/d/000000085/save-timing?orgId=1
    • Average and p95 time to parse a page

Details

SubjectRepoBranchLines +/-
Additional information about the ExternalData responsemediawiki/extensions/Kartographermaster+3 -1
Send event for every ExternalData expansionmediawiki/extensions/Kartographermaster+24 -6
Update event schema for Kartographer external dataschemas/event/secondarymaster+213 -5
New event schema for Kartographer ExternalData fetch performanceschemas/event/secondarymaster+220 -0
Log if external data expansion takes more than 1smediawiki/extensions/Kartographermaster+9 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

lilients_WMDE created this task.Jan 10 2023, 2:46 PM
lilients_WMDE moved this task from Backlog to In sprint on the WMDE-GeoInfo-FocusArea board.Jan 10 2023, 3:09 PM
lilients_WMDE mentioned this in T326317: Deploy geoshape expansion to wikis.Jan 12 2023, 2:22 PM
lilients_WMDE updated the task description. (Show Details)
awight added a parent task: T326953: Request for parser performance metrics.Jan 13 2023, 1:13 PM
gerritbot added a comment.Jan 16 2023, 2:33 PM

Change 880479 had a related patch set uploaded (by Awight; author: Awight):

[schemas/event/secondary@master] [WIP] New event schema for Kartographer ExternalData fetch performance

https://gerrit.wikimedia.org/r/880479

gerritbot added a project: Patch-For-Review.Jan 16 2023, 2:33 PM
Krinkle updated the task description. (Show Details)Jan 16 2023, 7:54 PM
awight updated the task description. (Show Details)Jan 17 2023, 11:22 AM
gerritbot added a comment.Jan 17 2023, 11:47 AM

Change 880937 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/extensions/Kartographer@master] [WIP] Send event for every ExternalData expansion

https://gerrit.wikimedia.org/r/880937

WMDE-Fisch added a project: WMDE-TechWish-Sprint-2023-01-18.Jan 18 2023, 9:11 AM
awight claimed this task.Jan 20 2023, 12:19 PM
awight moved this task from Sprint Backlog to Doing on the WMDE-TechWish-Sprint-2023-01-18 board.
awight renamed this task from Generate metrics for page saving and parsing times to Generate metrics for page saving and parsing times, to monitor impact of map external data expansion.Jan 20 2023, 12:21 PM
gerritbot added a comment.Jan 20 2023, 1:05 PM

Change 881871 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/extensions/Kartographer@master] Log if external data expansion takes more than 1s

https://gerrit.wikimedia.org/r/881871

gerritbot added a comment.Jan 20 2023, 1:28 PM

Change 881871 merged by jenkins-bot:

[mediawiki/extensions/Kartographer@master] Log if external data expansion takes more than 1s

https://gerrit.wikimedia.org/r/881871

ReleaseTaggerBot added a project: MW-1.40-notes (1.40.0-wmf.20; 2023-01-23).Jan 20 2023, 2:00 PM
awight updated the task description. (Show Details)Jan 20 2023, 2:18 PM
awight mentioned this in T323920: Deploy recent map server work to production ( Jan 2023 ).
gerritbot added a comment.Jan 23 2023, 11:41 AM

Change 880479 merged by jenkins-bot:

[schemas/event/secondary@master] New event schema for Kartographer ExternalData fetch performance

https://gerrit.wikimedia.org/r/880479

gerritbot added a comment.Jan 23 2023, 11:47 AM

Change 882628 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/extensions/Kartographer@master] Additional information about the ExternalData response

https://gerrit.wikimedia.org/r/882628

gerritbot added a comment.Jan 23 2023, 11:52 AM

Change 882631 had a related patch set uploaded (by Awight; author: Awight):

[schemas/event/secondary@master] Update event schema for Kartographer external data

https://gerrit.wikimedia.org/r/882631

awight removed awight as the assignee of this task.Jan 23 2023, 2:33 PM
awight updated the task description. (Show Details)
awight moved this task from Doing to Tech Review on the WMDE-TechWish-Sprint-2023-01-18 board.

There are still some gaps, but we can move this forward on a "good enough" basis.

gerritbot added a comment.Jan 27 2023, 12:01 PM

Change 882631 merged by jenkins-bot:

[schemas/event/secondary@master] Update event schema for Kartographer external data

https://gerrit.wikimedia.org/r/882631

gerritbot added a comment.Jan 29 2023, 9:08 AM

Change 880937 merged by jenkins-bot:

[mediawiki/extensions/Kartographer@master] Send event for every ExternalData expansion

https://gerrit.wikimedia.org/r/880937

gerritbot added a comment.Jan 29 2023, 9:08 AM

Change 882628 merged by jenkins-bot:

[mediawiki/extensions/Kartographer@master] Additional information about the ExternalData response

https://gerrit.wikimedia.org/r/882628

Maintenance_bot removed a project: Patch-For-Review.Jan 29 2023, 9:30 AM
ReleaseTaggerBot edited projects, added MW-1.40-notes (1.40.0-wmf.21; 2023-01-30); removed MW-1.40-notes (1.40.0-wmf.20; 2023-01-23).Jan 29 2023, 10:00 AM
awight moved this task from Tech Review to Demo on the WMDE-TechWish-Sprint-2023-01-18 board.Jan 30 2023, 9:19 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL