Page MenuHomePhabricator

MediaWiki Prometheus support
Open, MediumPublic

Description

As part of T228380, MediaWiki metrics need to make it into Prometheus somehow. There are a couple ways of doing this.

  1. Statsd Exporter -- This approach was attempted here.
  2. Native support -- This client library might help.

This task is complete when MediaWiki metrics are available in Prometheus.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 13 2019, 3:40 PM
colewhite triaged this task as Medium priority.Dec 13 2019, 3:40 PM
jcrespo moved this task from Backlog to Acknowledged on the Operations board.Dec 13 2019, 4:47 PM

My two cents re: prometheus_client_php current adapters, apcu and redis, since AIUI none are optimal/desirable. There might be another way inspired by what the ruby client does (see also this PromCon 2019 talk and video). Very broadly, the idea IIRC is for each process to mmap a separate file with its own metrics, then at collection time metrics are read from the files and merged.

We recently had a conversation about this.

  • There's no clear way to translate from an arbitrary string to a more Prometheus-friendly metric. Usage within MediaWiki is similar to: (StatsBuffer->{increment,gauge,timer}(string name, float|int val)) Example
    • This assumes we ignore setting statsd exporter in front of the output stream and translate each metric from . separated to _ separated. This loses the richness that labels gives us.

@Krinkle added:

a convention like metric_name.labels.go.here isn't enough as you wouldn't want prometheus to make them key_0, key_1 or something like that

  • Code will have to be written. Likely in the form of a formal metrics interface within MediaWiki Core.

@CDanis added:

we probably want the new API to encode some more semantic information somehow? probably, have some notion of what the 'base' metric name is, and what the labels are -- and some default way to sensibly flatten that into a statsd key. I think that makes more sense than starting from a statsd key and figuring out how to split out labels

One idea proposed to have some system find and expose the metrics MediaWiki would generate and dynamically create a statsd exporter config file.