Page MenuHomePhabricator

Frontend Save Timing broken (<1 metrics per minute)
Closed, ResolvedPublic

Description

https://grafana.wikimedia.org/dashboard/db/save-timing


https://grafana.wikimedia.org/dashboard/db/performance-metrics

Drop started straight at 2017-03-23 21:18:00 UTC

2017-03-23

  • 20:16 thcipriani@tin: rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.29.0-wmf.17

Details

Related Gerrit Patches:
mediawiki/extensions/NavigationTiming : wmf/1.29.0-wmf.17ext.NavigationTiming: Restore unsampled Save Timing
mediawiki/extensions/NavigationTiming : masterext.NavigationTiming: Restore unsampled Save Timing

Event Timeline

Krinkle created this task.Mar 25 2017, 4:37 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 25 2017, 4:37 AM

Most likely I broke something with 8957895a6e9e that somehow only affects Save Timing.

Yup. It used to be that navtiming is sampled but save timing is not sampled. After my change, both are sampled. The sampling is meant to apply to page views, not edits. This changed reduce Save Timing rate from 250/minute to 0 most minutes, and 1 or 2 in others.
https://grafana.wikimedia.org/dashboard/db/eventlogging-schema?var-schema=SaveTiming

I'll fix this on Monday.

Krinkle renamed this task from Frontend Save Timing broken (or regressed by +18.000s) to Frontend Save Timing broken (<1 metrics per minute).Mar 25 2017, 9:35 PM
Krinkle claimed this task.

Change 345051 had a related patch set uploaded (by Krinkle):
[mediawiki/extensions/NavigationTiming@master] ext.NavigationTiming: Restore unsampled Save Timing

https://gerrit.wikimedia.org/r/345051

Krinkle triaged this task as Unbreak Now! priority.Mar 27 2017, 9:42 PM
Restricted Application added subscribers: Jay8g, TerraCodes. · View Herald TranscriptMar 27 2017, 9:42 PM
Krinkle moved this task from Inbox to Doing on the Performance-Team board.Mar 27 2017, 9:42 PM

Change 345078 had a related patch set uploaded (by Krinkle):
[mediawiki/extensions/NavigationTiming@wmf/1.29.0-wmf.17] ext.NavigationTiming: Restore unsampled Save Timing

https://gerrit.wikimedia.org/r/345078

Change 345051 merged by jenkins-bot:
[mediawiki/extensions/NavigationTiming@master] ext.NavigationTiming: Restore unsampled Save Timing

https://gerrit.wikimedia.org/r/345051

Change 345078 merged by jenkins-bot:
[mediawiki/extensions/NavigationTiming@wmf/1.29.0-wmf.17] ext.NavigationTiming: Restore unsampled Save Timing

https://gerrit.wikimedia.org/r/345078

Mentioned in SAL (#wikimedia-operations) [2017-03-28T23:22:05Z] <thcipriani@tin> Synchronized php-1.29.0-wmf.17/extensions/NavigationTiming/modules/ext.navigationTiming.js: SWAT: [[gerrit:345078|ext.NavigationTiming: Restore unsampled Save Timing]] T161368 (duration: 00m 45s)

Krinkle closed this task as Resolved.Mar 28 2017, 11:40 PM