| Michael | |
| Jan 8 2025, 10:51 AM |
| F58257861: image.png | |
| Jan 23 2025, 11:59 AM |
| F58257604: image.png | |
| Jan 23 2025, 11:26 AM |
From the conversation in T355837#10438043, we learned that the unit to use with TimingMetric::observe is absolutely required to be milliseconds, despite the metric being keyed with the _seconds suffix, and despite us previously tracking our timings in seconds just fine.
That means we have to redo all our migrations for timing metrics, and the data collected until with staslib/prometheus now is useless.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T343020 Converting MediaWiki Metrics to StatsLib | |||
| Resolved | herron | T350591 Audit legacy mediawiki stats used in production dashboards | |||
| Open | None | T350592 EPIC: migrate in use metrics and dashboards to statslib | |||
| Resolved | DMburugu | T367429 [Epic] FY 2024-25 Growth Maintenance Work | |||
| Open | None | T308313 Use PrefixingStatsdDataFactoryProxy for suggested edits timings | |||
| Open | None | T375600 Create Grafana panel with percentage of (mobile) homepage views with no available tasks | |||
| Stalled | None | T397606 Remove Graphite queries from Growth Grafana dashboards | |||
| Resolved | None | T359352 Migrate GrowthExperiments Extension to statslib | |||
| Resolved | Michael | T383208 StatsLib timings MUST be recorded as milliseconds |
Change #1109051 had a related patch set uploaded (by Michael Große; author: Michael Große):
[mediawiki/extensions/GrowthExperiments@master] fix(tracking): TimingMetric:observe records milliseconds
Change #1109051 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] fix(tracking): TimingMetric:observe records milliseconds
Change #1111196 had a related patch set uploaded (by Michael Große; author: Michael Große):
[mediawiki/extensions/GrowthExperiments@wmf/1.44.0-wmf.12] fix(tracking): TimingMetric:observe records milliseconds
Change #1111196 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@wmf/1.44.0-wmf.12] fix(tracking): TimingMetric:observe records milliseconds
Mentioned in SAL (#wikimedia-operations) [2025-01-14T15:33:41Z] <Lucas_WMDE> previous deployment also included [[gerrit:rGERRIT111119698026|fix(tracking): TimingMetric:observe records milliseconds]] (T383208)
@Michael Is there anything else we need to do in this task? Should it still be in Incoming?
Mh, for practical purposes I would want to wait until tomorrow when we have hopefully some sensible data. Then we should be able to see if the changes here had the desired effects and decide on next steps. But I can move it to doing just as well.
I started looking into how to migrate one of our top performance metrics for Special:Homepage, "$platform rendering speed Special:Homepage (p99)", in this case I started with desktop at the platform.
But so far the data does not match at all:
The query that is being used for Graphite:
aliasByNode(MediaWiki.timing.growthExperiments.specialHomepage.serverSideRender.desktop.p99, 4, 5, 6)
The query that is being used for Prometheus:
histogram_quantile(0.99, sum by(le) ( rate(mediawiki_GrowthExperiments_special_homepage_server_side_render_seconds_bucket{platform="desktop"}[5m]) ))
Not sure what I'm doing wrong.
Change #1113796 had a related patch set uploaded (by Michael Große; author: Michael Große):
[mediawiki/extensions/GrowthExperiments@master] fix: track correct data in legacy Graphite query
Change #1113796 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] fix: track correct data in legacy Graphite query
Status update: currently working on creating a visualization and showing that on a dashboard
This can now be considered done. I've migrated two panels on the Special:Homepage / Suggested Edits dashboard (the first two under "image recommendation service"), and the data is compatible with the data we used to get from Graphite.
Also, I wrote down some basic notes for this kind of dashboard on https://wikitech.wikimedia.org/wiki/Grafana/Best_practices/Getting_Started_with_Thanos_panels#Timings