The Observability team proposes a "value-first" approach for the migration process. This means we will prioritize migrating "high value" targets first, covering the most frequently used metrics and dashboards before moving on to less commonly used ones. However, before proceeding with the migration, we need to have an initial snapshot of what this approach entails as of 2024-01-08
Please note the following steps to follow roughly per every conversion:
1. Identify the metric (or group of metrics) that will be to be converted.
2. Create/assign a Phabricator subtask linked to this task (with granularity of individual metric or group of metrics) and update task description to reflect which task(s) have been created for which metric(s).
3. Follow the migration process as outlined below.
4. Secure/Conduct code review(s).
5. Deploy the changes to production via the train (https://wikitech.wikimedia.org/wiki/Deployments/Train).
6. Verify that the changes have been successfully implemented.
7. Place the metrics subtask in a 2-3 week waiting period to allow prometheus time to establish 2-3 weeks of metric history
8. After 2-3 week waiting period is complete, update the dashboard:
* Save a copy of the dashboard using legacy metrics as-is into the Legacy grafana dashboard folder
* Replace the old Graphite metric(s) with the new Prometheus metric(s) and save/update the live dashboard
Please follow the guidelines and standards outlined in the provided documentation:
- https://www.mediawiki.org/wiki/Manual:Stats for detailed guidance on the conversion process.
- https://drive.google.com/file/d/12yQEuOapkML1vb9MgCaX1QzbLBdXE6X2/view for a video tutorial on the conversion process.
- https://docs.google.com/presentation/d/1SZWf_D3mWNX-XHN8PHYI84LDZr6GUQC2AMhZ9mQXCI0/edit#slide=id.g2795460c956_0_23 for slides on the best practices for converting metrics to statslib.
Full List of MediaWiki Metrics used in Grafana Dashboards as of 2023-01-08: P54551
Overall Grafana Graphite datasource utilization, count by usage in queries P54396
--**[[https://grafana.wikimedia.org/d/nCxX65cSk/mediawiki-statslib-migration?orgId=1 | Migration Progress Dashboard]]** ---** [[https://grafana.wikimedia.org/d/K6DEOo5Ik/grafana-graphite-datasource-utilization?orgId=1 | Dashboards using Graphite]]**--
**Metrics to Migrate**
Below is a full list of graphite metrics to be migrated, grouped by component. Please claim the task for ongoing work or any metrics/components under your care and proceed with the migration; if you need help, please contact #sre_observability.
**[[ https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?title=Migrate%20%3Cmetric(s)%3E%20to%20statslib&description=Follow%20the%20migration%20process%20as%20outlined%20below.%0D%0A%0D%0ASecure%2FConduct%20code%20review(s).%0D%0ADeploy%20the%20changes%20to%20production%20via%20the%20train%20(https%3A%2F%2Fwikitech.wikimedia.org%2Fwiki%2FDeployments%2FTrain).%0D%0AVerify%20that%20the%20changes%20have%20been%20successfully%20implemented.%0D%0AUpdate%20the%20relevant%20dashboard(s)%20by%20replacing%20the%20old%20Graphite%20metric(s)%20with%20the%20new%20Prometheus%20metric(s).%0D%0APlease%20follow%20the%20guidelines%20and%20standards%20outlined%20in%20the%20provided%20documentation%3A%0D%0A%0D%0Ahttps%3A%2F%2Fwww.mediawiki.org%2Fwiki%2FManual%3AStats%20for%20detailed%20guidance%20on%20the%20conversion%20process.%0D%0Ahttps%3A%2F%2Fdrive.google.com%2Ffile%2Fd%2F12yQEuOapkML1vb9MgCaX1QzbLBdXE6X2%2Fview%20for%20a%20video%20tutorial%20on%20the%20conversion%20process.%0D%0Ahttps%3A%2F%2Fdocs.google.com%2Fpresentation%2Fd%2F1SZWf_D3mWNX-XHN8PHYI84LDZr6GUQC2AMhZ9mQXCI0%2Fedit%23slide%3Did.g2795460c956_0_23%20for%20slides%20on%20the%20best%20practices%20for%20converting%20metrics%20to%20statslib.&projects=Observability-Metrics&priority=normal&parent=350592 | Please use this template when creating metric migration subtasks ]]**
Core:
* {T354905}
* {T359236}
* {T356812}
* {T356814}
* {T356815}
* {T359235}
* {T359240}
* {T359241}
* {T359242}
* {T359243}
Performance:
* {T356817}
ContentTranslation:
* {T359237}
* {T359238}
* {T359239}
RevisionSlider:
* {T354907}
* {T359245}
* {T359244}
Wikibase:
* {T354909}
* {T359247}
* {T359246}
* {T359248}
* {T359249}
* {T359250}
* {T359251}
* {T359252}
* {T359253}
* {T359254}
Parsoid:
* {T354908}
* {T359453}
* {T359454}
* {T359455}
* {T359457}
* {T359402}
* {T359395}
* {T359394}
* {T359393}
* {T359392}
* {T359391}
* {T359390}
* {T359389}
* {T359388}
Resourceloader:
* {T355960}
* {T359396}
* {T359397}
* {T359398}
TwoColConflict:
* {T359261}
* {T359262}
Centralauth:
* {T359263}
Authmanager
* {T359264}
Betafeatures:
* {T359339}
CirrusSearch:
* {T359033}
AdvancedSearch:
* {T359342}
* {T359341}
Media:
* {T359344}
* {T359345}
Echo:
* {T359346}
* {T359347}
Catwatch:
* {T359349}
* {T359350}
GrowthExperiments:
* {T359351}
* {T359352}
* {T359353}
* {T359354}
* {T359355}
* {T359356}
MediaModeration:
* {T359357}
Abusefilter:
* {T359359}
* {T359360}
* {T359361}
* {T359362}
AdvancedSearch:
* {T359363}
API:
* {T359383}
Arclamp:
* {T359385}
Articleplaceholder:
* {T359386}
BlockNotices:
* {T359458}
Bouncehandler:
* {T359459}
Chrome:
* {T359460}
CodeMirror:
* {T359461}
Cognate:
* {T359462}
Discussiontools:
* {T359463}
Editor:
* {T359464}
Editstash:
* {T359465}
Electronpdf:
* {T359466}
* {T359467}
* {T359468}
* {T359470}
PageTriage:
* {T359471}
Phonos:
* {T359485}
ExternalGuidance:
* {T359484}
FileImporter:
* {T359481}
Firefox"
* {T359482}
Global_Blocking:
* {T359480}
Globalwatchlist:
* {T359479}
* {T359478}
Html2wt:
* {T359477}
Html:
* {T359475}
* {T359476}
ipinfo:
* {T359474}
Jobexecutor:
* {T359473}
Jobqueue:
* {T359472}
Loadbalancer:
* {T359382}
Loginnotify:
* {T359381}
Mediamoderation:
* {T359380}
Media:
* {T359379}
Minerva:
* {T359377}
Mobile:
* {T359378}
Objectcache:
* {T359376}
Pagedtiffhandler:
* {T359373}
PageStore:
* {T359374}
ParserOutput:
* {T359372}
Pcache:
* {T359368}
* {T359370}
RateLimiter:
* {T359367}
Rdbms_trxprofiler:
* {T359366}
Refreshlinks:
* {T359365}
Rest_api:
* {T359364}
RevisionOutputCache:
* {T359278}
RevisionSlider:
* {T359277}
Score:
* {T359276}
Scribunto:
* {T359275}
Setup_load_schema:
* {T359274}
Spamblock:
* {T359273}
Stable_pcache:
* {T359272}
Syntashilight:
* {T359270}
TemplateData:
* {T359271}
TemplateWizard:
* {T359269}
Timeline_error:
* {T359268}
Timing:
* {T359267}
Tmp_settings_load:
* {T359266}
Translate:
* {T359265}
TTFE:
* {T359260}
VisualEditor:
* {T359259}
* {T359258}
Wanobjectcache:
* {T359257}
WatchedItemStore:
* {T359256}
Wmfstatic
* {T359255}