Page MenuHomePhabricator
Create Task
Maniphest T240685

MediaWiki Prometheus support
Closed, ResolvedPublic
Actions

Assigned To
colewhite
Authored By
colewhite
Dec 13 2019, 3:40 PM
Tags
Referenced Files
None
Subscribers
Aklapper
andrea.denisse
CDanis
ChristianHeusel
colewhite
• dpifke
fgiunchedi
View All 17 Subscribers
Tokens
"Love" token, awarded by lmata.

Description

As part of T228380, MediaWiki metrics need to make it into Prometheus.

This task is complete when:

  • MediaWiki is capable of exporting Prometheus-style metrics
  • These metrics are consumed by the production Prometheus
  • All MediaWiki metrics and module metrics are shipped in this manner
  • All dashboards are updated to use these new metrics
  • All previous code changes needed for this are merged and deployed in production.

High level checklist:

Details

SubjectRepoBranchLines +/-
Update wgStatsTarget to port 9125operations/mediawiki-configmaster+1 -1
Enable $wgStatsTarget for requests to mwdebugoperations/mediawiki-configmaster+9 -2
mediawiki: add statsd exporteroperations/deployment-chartsmaster+36 -0
mw-debug: add statsd-exporteroperations/deployment-chartsmaster+39 -0
profile: provision statsd_exporter on canary_appserveroperations/puppetproduction+8 -1
MediaWiki.php: Convert executeTiming metric to new Stats librarymediawiki/coremaster+14 -4
Stats: make component optionalmediawiki/coremaster+124 -101
Metrics: expand Sample instance parametersmediawiki/coremaster+18 -27
Stats: add copy to statsd featuremediawiki/coremaster+106 -3
Stats: add static labels featuremediawiki/coremaster+132 -29
Stats: use MetricsInterface where neededmediawiki/coremaster+14 -15
Stats: add timing start and stop helper functionsmediawiki/coremaster+33 -0
Stats: simplify metrics configuration, enforce builder patternmediawiki/coremaster+128 -306
Metrics: implement MetricsCache classmediawiki/coremaster+207 -100
Metrics: simplify MetricUtils, introduce BaseMetricInterfacemediawiki/coremaster+887 -401
Metrics: refactor rendering interfacemediawiki/coremaster+594 -139
tests: Move stats library into Stats foldermediawiki/coremaster+0 -0
Stats: move stats library into Stats foldermediawiki/coremaster+6 -6
Metrics: rename Metrics lib to Statsmediawiki/coremaster+436 -431
Metrics: refactor emitter instantiationmediawiki/coremaster+347 -495
Metrics: move format configuration to separate classmediawiki/coremaster+64 -14
Metrics: add metric property accessorsmediawiki/coremaster+92 -0
Metrics: move metric implementations to subdirectorymediawiki/coremaster+32 -16
Metrics: rename extension to componentmediawiki/coremaster+65 -65
Metrics: Minor doc improvements and announce feature in release notesmediawiki/coremaster+40 -34
Metrics: Add test coverage for getRenderedSamples() and send()mediawiki/coremaster+75 -69
profile: add enable_relay flag to statsd exporter profileoperations/puppetproduction+8 -1
role: install statsd_exporter on canary appserversoperations/puppetproduction+4 -0
Metrics: reduce the number of packets sent by MetricsFactory->flush()mediawiki/coremaster+19 -4
Metrics: Wire up MetricsFactory into ServiceWiring and emit stepsmediawiki/coremaster+60 -0
Metrics: Wire up MetricsFactory into ServiceWiringmediawiki/coremaster+22 -0
Metrics: Perform MetricsFactory->flush() in emitBufferedStatsdData()mediawiki/coremaster+12 -2
Metrics: Implement statsd-exporter compatible Metrics interfacemediawiki/coremaster+1 K -0
Metrics: Implement and enable statsd-exporter compatible Metrics interfacemediawiki/coremaster+996 -7
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
gerritbot added a comment.Nov 9 2022, 6:41 PM

Change 854630 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: rename extension to component

https://gerrit.wikimedia.org/r/854630

gerritbot added a comment.Nov 9 2022, 7:08 PM

Change 854631 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: expand Sample instance parameters

https://gerrit.wikimedia.org/r/854631

gerritbot added a comment.Nov 9 2022, 8:11 PM

Change 854632 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: move metric implementations to subdirectory

https://gerrit.wikimedia.org/r/854632

gerritbot added a comment.Nov 9 2022, 10:01 PM

Change 855127 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: add metric property accessors

https://gerrit.wikimedia.org/r/855127

gerritbot added a comment.Nov 9 2022, 10:20 PM

Change 855128 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: implement MetricsCache

https://gerrit.wikimedia.org/r/855128

gerritbot added a comment.Nov 10 2022, 12:18 AM

Change 855129 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: move format configuration to separate class

https://gerrit.wikimedia.org/r/855129

gerritbot added a comment.Nov 10 2022, 12:43 AM

Change 855130 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: refactor rendering interface

https://gerrit.wikimedia.org/r/855130

gerritbot added a comment.Nov 10 2022, 3:28 PM

Change 855626 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: simplify MetricUtils, introduce BaseMetricInterface

https://gerrit.wikimedia.org/r/855626

gerritbot added a comment.Nov 10 2022, 3:56 PM

Change 855627 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: wire up MetricsUDPEmitter

https://gerrit.wikimedia.org/r/855627

gerritbot added a comment.Nov 11 2022, 3:01 AM

Change 855633 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: simplify metrics configuration, enforce builder pattern

https://gerrit.wikimedia.org/r/855633

gerritbot added a comment.Nov 16 2022, 7:46 PM

Change 857057 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: use MetricsInterface where needed

https://gerrit.wikimedia.org/r/857057

gerritbot added a comment.Nov 16 2022, 8:28 PM

Change 857060 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: add timing start and stop helper functions

https://gerrit.wikimedia.org/r/857060

gerritbot added a comment.Nov 16 2022, 9:11 PM

Change 857061 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: add static labels feature

https://gerrit.wikimedia.org/r/857061

gerritbot added a comment.Nov 16 2022, 9:50 PM

Change 857062 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: add copy to statsd feature

https://gerrit.wikimedia.org/r/857062

gerritbot added a comment.Nov 17 2022, 7:11 PM

Change 854630 merged by jenkins-bot:

[mediawiki/core@master] Metrics: rename extension to component

https://gerrit.wikimedia.org/r/854630

gerritbot added a comment.Nov 17 2022, 7:19 PM

Change 854631 merged by jenkins-bot:

[mediawiki/core@master] Metrics: expand Sample instance parameters

https://gerrit.wikimedia.org/r/854631

gerritbot added a comment.Nov 17 2022, 7:19 PM

Change 854632 merged by jenkins-bot:

[mediawiki/core@master] Metrics: move metric implementations to subdirectory

https://gerrit.wikimedia.org/r/854632

gerritbot added a comment.Nov 17 2022, 7:20 PM

Change 855127 merged by jenkins-bot:

[mediawiki/core@master] Metrics: add metric property accessors

https://gerrit.wikimedia.org/r/855127

ReleaseTaggerBot added a project: MW-1.40-notes (1.40.0-wmf.12; 2022-11-28).Nov 17 2022, 8:00 PM
Tgr mentioned this in T326607: Future of liuggio/statsd-php-client?.Jan 11 2023, 3:28 AM
Michael subscribed.Jan 18 2023, 12:37 PM
gerritbot added a comment.Feb 22 2023, 12:41 AM

Change 855128 merged by jenkins-bot:

[mediawiki/core@master] Metrics: implement MetricsCache class

https://gerrit.wikimedia.org/r/855128

ReleaseTaggerBot edited projects, added MW-1.40-notes (1.40.0-wmf.25; 2023-02-27); removed MW-1.40-notes (1.40.0-wmf.12; 2022-11-28).Feb 22 2023, 1:00 AM
gerritbot added a comment.Feb 23 2023, 12:03 AM

Change 855129 merged by jenkins-bot:

[mediawiki/core@master] Metrics: move format configuration to separate class

https://gerrit.wikimedia.org/r/855129

gerritbot added a comment.Feb 23 2023, 12:11 AM

Change 855130 merged by jenkins-bot:

[mediawiki/core@master] Metrics: refactor rendering interface

https://gerrit.wikimedia.org/r/855130

gerritbot added a comment.Feb 23 2023, 12:41 AM

Change 855626 merged by jenkins-bot:

[mediawiki/core@master] Metrics: simplify MetricUtils, introduce BaseMetricInterface

https://gerrit.wikimedia.org/r/855626

gerritbot added a comment.Feb 24 2023, 10:31 PM

Change 891868 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Metrics: rename Metrics lib to Stats

https://gerrit.wikimedia.org/r/891868

gerritbot added a comment.Feb 25 2023, 11:20 PM

Change 855627 merged by jenkins-bot:

[mediawiki/core@master] Metrics: refactor emitter instantiation

https://gerrit.wikimedia.org/r/855627

gerritbot added a comment.Feb 25 2023, 11:20 PM

Change 891868 merged by jenkins-bot:

[mediawiki/core@master] Metrics: rename Metrics lib to Stats

https://gerrit.wikimedia.org/r/891868

gerritbot added a comment.Feb 27 2023, 4:42 PM

Change 891882 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Stats: move stats library into Stats folder

https://gerrit.wikimedia.org/r/891882

gerritbot added a comment.Feb 27 2023, 5:09 PM

Change 891882 merged by jenkins-bot:

[mediawiki/core@master] Stats: move stats library into Stats folder

https://gerrit.wikimedia.org/r/891882

gerritbot added a comment.Feb 28 2023, 8:39 PM

Change 893056 had a related patch set uploaded (by Umherirrender; author: Umherirrender):

[mediawiki/core@master] tests: Move stats library into Stats folder

https://gerrit.wikimedia.org/r/893056

gerritbot added a comment.Feb 28 2023, 8:57 PM

Change 893056 merged by jenkins-bot:

[mediawiki/core@master] tests: Move stats library into Stats folder

https://gerrit.wikimedia.org/r/893056

ReleaseTaggerBot edited projects, added MW-1.40-notes (1.40.0-wmf.26; 2023-03-06); removed MW-1.40-notes (1.40.0-wmf.25; 2023-02-27).Feb 28 2023, 9:00 PM
gerritbot added a comment.Mar 6 2023, 8:10 PM

Change 894577 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Stats: make component optional

https://gerrit.wikimedia.org/r/894577

gerritbot added a comment.Mar 6 2023, 11:31 PM

Change 855633 merged by jenkins-bot:

[mediawiki/core@master] Stats: simplify metrics configuration, enforce builder pattern

https://gerrit.wikimedia.org/r/855633

gerritbot added a comment.Mar 6 2023, 11:31 PM

Change 857057 merged by jenkins-bot:

[mediawiki/core@master] Stats: use MetricsInterface where needed

https://gerrit.wikimedia.org/r/857057

gerritbot added a comment.Mar 6 2023, 11:31 PM

Change 857060 merged by jenkins-bot:

[mediawiki/core@master] Stats: add timing start and stop helper functions

https://gerrit.wikimedia.org/r/857060

gerritbot added a comment.Mar 6 2023, 11:33 PM

Change 857061 merged by jenkins-bot:

[mediawiki/core@master] Stats: add static labels feature

https://gerrit.wikimedia.org/r/857061

gerritbot added a comment.Mar 10 2023, 9:48 PM

Change 857062 merged by jenkins-bot:

[mediawiki/core@master] Stats: add copy to statsd feature

https://gerrit.wikimedia.org/r/857062

ReleaseTaggerBot edited projects, added MW-1.40-notes (1.40.0-wmf.27; 2023-03-13); removed MW-1.40-notes (1.40.0-wmf.26; 2023-03-06).Mar 10 2023, 10:00 PM
gerritbot added a comment.Mar 17 2023, 5:27 PM

Change 900706 had a related patch set uploaded (by Cwhite; author: Cwhite):

[operations/puppet@production] mediawiki: provision statsd_exporter on canary_appserver

https://gerrit.wikimedia.org/r/900706

Krinkle mentioned this in T321634: Re-evaluate need for "cool-off bounce" in WANObjectCache.Apr 17 2023, 6:41 AM
Addshore unsubscribed.Jun 27 2023, 12:39 PM
andrea.denisse subscribed.Jul 13 2023, 10:43 PM
lmata added a project: SRE Observability (FY2023/2024-Q1).Jul 28 2023, 5:33 PM
lmata moved this task from Inbox to In progress on the SRE Observability (FY2023/2024-Q1) board.Jul 28 2023, 5:35 PM
lmata mentioned this in T343020: Converting MediaWiki Metrics to StatsLib.Jul 28 2023, 5:45 PM
colewhite added a subtask: T344748: MediaWiki Core - Review and merge StatsLib patch.Aug 22 2023, 5:02 PM
colewhite added a subtask: T344751: Decide on default histogram buckets for MediaWiki timers.Aug 22 2023, 5:48 PM
herron subscribed.Aug 23 2023, 1:43 PM
Krinkle closed subtask T344748: MediaWiki Core - Review and merge StatsLib patch as Resolved.Sep 4 2023, 5:02 PM
gerritbot added a comment.Sep 4 2023, 5:17 PM

Change 894577 merged by jenkins-bot:

[mediawiki/core@master] Stats: make component optional

https://gerrit.wikimedia.org/r/894577

ReleaseTaggerBot added a project: MW-1.41-notes (1.41.0-wmf.25; 2023-09-05).Sep 4 2023, 6:00 PM
gerritbot added a comment.Sep 7 2023, 9:56 PM

Change 955015 had a related patch set uploaded (by Cwhite; author: Cwhite):

[operations/mediawiki-config@master] Add StatsLib settings for Test env

https://gerrit.wikimedia.org/r/955015

gerritbot added a comment.Sep 7 2023, 10:02 PM

Change 955016 had a related patch set uploaded (by Cwhite; author: Cwhite):

[mediawiki/core@master] Convert executeTiming to StatsLib

https://gerrit.wikimedia.org/r/955016

gerritbot added a comment.Sep 25 2023, 8:44 PM

Change 955016 merged by jenkins-bot:

[mediawiki/core@master] MediaWiki.php: Convert executeTiming metric to new Stats library

https://gerrit.wikimedia.org/r/955016

ReleaseTaggerBot edited projects, added MW-1.41-notes (1.41.0-wmf.28; 2023-09-26); removed MW-1.41-notes (1.41.0-wmf.25; 2023-09-05).Sep 25 2023, 9:00 PM
Tgr mentioned this in T348206: Improve logging, monitoring and test coverage for MediaWiki Platform team authentication extensions.Oct 5 2023, 12:52 AM
lmata edited projects, added SRE Observability (FY2023/2024-Q2); removed SRE Observability (FY2023/2024-Q1).Oct 9 2023, 2:30 PM
herron closed subtask T344751: Decide on default histogram buckets for MediaWiki timers as Resolved.Oct 31 2023, 1:34 PM
herron updated the task description. (Show Details)Oct 31 2023, 2:03 PM
herron updated the task description. (Show Details)Oct 31 2023, 2:20 PM
gerritbot added a comment.Oct 31 2023, 8:32 PM

Change 900706 abandoned by Cwhite:

[operations/puppet@production] profile: provision statsd_exporter on canary_appserver

Reason:

superseded by T345377

https://gerrit.wikimedia.org/r/900706

gerritbot added a comment.Nov 7 2023, 9:52 AM

Change 972342 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/deployment-charts@master] mw-debug: add statsd-exporter

https://gerrit.wikimedia.org/r/972342

gerritbot added a comment.Nov 7 2023, 9:52 AM

Change 972343 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/deployment-charts@master] mediawiki: add statsd exporter

https://gerrit.wikimedia.org/r/972343

gerritbot added a comment.Nov 8 2023, 5:07 PM

Change 972342 merged by jenkins-bot:

[operations/deployment-charts@master] mw-debug: add statsd-exporter

https://gerrit.wikimedia.org/r/972342

gerritbot added a comment.Nov 9 2023, 8:05 AM

Change 972343 merged by jenkins-bot:

[operations/deployment-charts@master] mediawiki: add statsd exporter

https://gerrit.wikimedia.org/r/972343

gerritbot added a comment.Dec 13 2023, 10:05 PM

Change 955015 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable $wgStatsTarget for requests to mwdebug

https://gerrit.wikimedia.org/r/955015

Stashbot added a comment.Dec 13 2023, 10:05 PM

Mentioned in SAL (#wikimedia-operations) [2023-12-13T22:05:49Z] <jhuneidi@deploy2002> Started scap: Backport for [[gerrit:982857|Partially undeploy Reader Demographics 2 survey (T344393)]], [[gerrit:955015|Enable $wgStatsTarget for requests to mwdebug (T240685)]]

Stashbot mentioned this in T344393: Quicksurvey deployment for readers survey .Dec 13 2023, 10:05 PM

Mentioned in SAL (#wikimedia-operations) [2023-12-13T22:07:13Z] <jhuneidi@deploy2002> dani and jhuneidi and cwhite: Backport for [[gerrit:982857|Partially undeploy Reader Demographics 2 survey (T344393)]], [[gerrit:955015|Enable $wgStatsTarget for requests to mwdebug (T240685)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Stashbot added a comment.Dec 13 2023, 10:18 PM

Mentioned in SAL (#wikimedia-operations) [2023-12-13T22:18:22Z] <jhuneidi@deploy2002> Finished scap: Backport for [[gerrit:982857|Partially undeploy Reader Demographics 2 survey (T344393)]], [[gerrit:955015|Enable $wgStatsTarget for requests to mwdebug (T240685)]] (duration: 12m 33s)

gerritbot added a comment.Dec 13 2023, 10:20 PM

Change 982867 had a related patch set uploaded (by Cwhite; author: Cwhite):

[operations/mediawiki-config@master] Update wgStatsTarget to port 9125

https://gerrit.wikimedia.org/r/982867

gerritbot added a comment.Dec 13 2023, 10:47 PM

Change 982867 merged by jenkins-bot:

[operations/mediawiki-config@master] Update wgStatsTarget to port 9125

https://gerrit.wikimedia.org/r/982867

Stashbot added a comment.Dec 13 2023, 10:48 PM

Mentioned in SAL (#wikimedia-operations) [2023-12-13T22:48:01Z] <jhuneidi@deploy2002> Started scap: Backport for [[gerrit:982867|Update wgStatsTarget to port 9125 (T240685)]], [[gerrit:982925|[BC] Enable desktop diff and history pages on mobile (T350181 T353388)]]

Stashbot mentioned this in T350181: Enable desktop diff page on mobile site.Dec 13 2023, 10:48 PM
Stashbot mentioned this in T353388: Enable desktop history HTML on mobile.

Mentioned in SAL (#wikimedia-operations) [2023-12-13T22:49:33Z] <jhuneidi@deploy2002> jhuneidi and jdlrobson and cwhite: Backport for [[gerrit:982867|Update wgStatsTarget to port 9125 (T240685)]], [[gerrit:982925|[BC] Enable desktop diff and history pages on mobile (T350181 T353388)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Stashbot added a comment.Dec 13 2023, 10:57 PM

Mentioned in SAL (#wikimedia-operations) [2023-12-13T22:57:43Z] <jhuneidi@deploy2002> Finished scap: Backport for [[gerrit:982867|Update wgStatsTarget to port 9125 (T240685)]], [[gerrit:982925|[BC] Enable desktop diff and history pages on mobile (T350181 T353388)]] (duration: 09m 42s)

Maintenance_bot removed a project: Patch-For-Review.Dec 13 2023, 11:10 PM
ChristianHeusel subscribed.Jan 10 2024, 5:54 PM
lmata updated the task description. (Show Details)Jan 12 2024, 5:12 PM
lmata edited projects, added SRE Observability (FY2023/2024-Q3); removed SRE Observability (FY2023/2024-Q2).Jan 12 2024, 5:38 PM
lmata moved this task from Inbox to Epics In Progress on the SRE Observability (FY2023/2024-Q3) board.
colewhite moved this task from Backlog to Epics in Progress on the MediaWiki-libs-Stats board.Jan 19 2024, 9:14 PM
colewhite edited projects, added SRE Observability (FY2023/2024-Q4); removed SRE Observability (FY2023/2024-Q3).Apr 3 2024, 10:45 PM
colewhite moved this task from Inbox to Epics In Progress on the SRE Observability (FY2023/2024-Q4) board.Apr 3 2024, 11:09 PM
lmata updated the task description. (Show Details)Apr 4 2024, 8:31 PM
colewhite closed subtask T249164: RFC: Better interface for generating metrics in MediaWiki as Resolved.Apr 4 2024, 8:35 PM
colewhite removed colewhite as the assignee of this task.Apr 15 2024, 2:42 PM
colewhite closed this task as Resolved.Apr 15 2024, 3:12 PM
colewhite claimed this task.

This epic will continue in T343020: Converting MediaWiki Metrics to StatsLib

Tarrow mentioned this in T364107: Collect metrics from Mediawiki with Prometheus.May 3 2024, 11:08 PM
lmata mentioned this in T228380: Tech debt: sunsetting of Graphite.Mon, May 20, 3:54 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL