Page MenuHomePhabricator

EPIC: migrate in use metrics and dashboards to statslib
Open, HighPublic

Description

The Observability team proposes a "value-first" approach for the migration process. This means we will prioritize migrating "high value" targets first, covering the most frequently used metrics and dashboards before moving on to less commonly used ones. However, before proceeding with the migration, we need to have an initial snapshot of what this approach entails as of 2024-01-08

Please note the following steps to follow roughly per every conversion:

  1. Identify the metric (or group of metrics) that will be to be converted.
  2. Create/assign a Phabricator subtask linked to this task (with granularity of individual metric or group of metrics) and update task description to reflect which task(s) have been created for which metric(s).
  3. Follow the migration process as outlined below.
  4. Secure/Conduct code review(s).
  5. Deploy the changes to production via the train (https://wikitech.wikimedia.org/wiki/Deployments/Train).
  6. Verify that the changes have been successfully implemented.
  7. Place the metrics subtask in a 2-3 week waiting period to allow prometheus time to establish 2-3 weeks of metric history
  8. After 2-3 week waiting period is complete, update the dashboard:
    • Save a copy of the dashboard using legacy metrics as-is into the Legacy grafana dashboard folder
    • Replace the old Graphite metric(s) with the new Prometheus metric(s) and save/update the live dashboard

Please follow the guidelines and standards outlined in the provided documentation:

Full List of MediaWiki Metrics used in Grafana Dashboards as of 2023-01-08: P54551

Overall Grafana Graphite datasource utilization, count by usage in queries P54396

--Migration Progress Dashboard --- Dashboards using Graphite--

Metrics to Migrate

Below is a full list of graphite metrics to be migrated, grouped by component. Please claim the task for ongoing work or any metrics/components under your care and proceed with the migration; if you need help, please contact SRE Observability.

Please use this template when creating metric migration subtasks

Core:

Performance:

ContentTranslation:

Collection:

RevisionSlider:

Wikibase:

Parsoid:

Resourceloader:

TwoColConflict:

Centralauth:

Authmanager

Betafeatures:

CirrusSearch:

AdvancedSearch:

Media:

Echo:

Catwatch:

GrowthExperiments:

MediaModeration:

Abusefilter:

AdvancedSearch:

API:

Arclamp:

Articleplaceholder:

BlockNotices:

Bouncehandler:

Chrome:

CodeMirror:

Cognate:

Discussiontools:

Editor:

Editstash:

Electronpdf:

PageTriage:

Phonos:

ExternalGuidance:

FileImporter:

Firefox"

Global_Blocking:

Globalwatchlist:

Html2wt:

Html:

ipinfo:

Jobexecutor:

Jobqueue:

Loadbalancer:

Loginnotify:

Mediamoderation:

Media:

Minerva:

Mobile:

Objectcache:

Pagedtiffhandler:

PageStore:

ParserOutput:

Pcache:

RateLimiter:

Rdbms_trxprofiler:

Refreshlinks:

Rest_api:

RevisionOutputCache:

RevisionSlider:

Score:

Scribunto:

Setup_load_schema:

Spamblock:

Stable_pcache:

Syntashilight:

TemplateData:

TemplateWizard:

Timeline_error:

Timing:

Tmp_settings_load:

Translate:

TTFE:

VisualEditor:

Wanobjectcache:

WatchedItemStore:

Wmfstatic

Related Objects

StatusSubtypeAssignedTask
OpenNone
Resolvedherron
OpenNone
DuplicateDAlangi_WMF
DuplicateNone
OpenNone
OpenNone
Resolvedcolewhite
ResolvedDAlangi_WMF
OpenNone
Resolvedcolewhite
In Progresscolewhite
Resolvedcolewhite
DuplicateNone
OpenDAlangi_WMF
OpenNone
ResolvedDAlangi_WMF
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
Resolvedcolewhite
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
In Progressandrea.denisse
OpenNone
OpenNone
OpenNone
OpenNone
Resolvedcolewhite
Resolvedcolewhite
Resolvedcolewhite
In ProgressDAlangi_WMF
Resolvedcolewhite
OpenNone
Resolvedcolewhite
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
DuplicateNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
Resolvedcolewhite
OpenNone
Resolvedcolewhite
OpenNone
OpenNone
Resolvedcolewhite
OpenNone
Resolvedcolewhite
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
DuplicateNone
Resolvedlmata
DuplicateNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
Resolvedcolewhite
InvalidNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 990176 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] ApiMain, ApiQuery: update stats to generate StatsFactory metrics

https://gerrit.wikimedia.org/r/990176

Change 990176 merged by jenkins-bot:

[mediawiki/core@master] ApiMain, ApiQuery: update stats to generate StatsFactory metrics

https://gerrit.wikimedia.org/r/990176

herron updated the task description. (Show Details)
lmata updated the task description. (Show Details)
lmata renamed this task from migrate in use metrics and dashboards to statslib to EPIC: migrate in use metrics and dashboards to statslib.Mar 6 2024, 1:49 PM
lmata updated the task description. (Show Details)