[[ https://www.mediawiki.org/wiki/ResourceLoader/Core_modules#mw.track | mw.track ]] 'stats.' topic prefix is a MediaWiki JavaScript API that is used to submit operational metrics from browsers to Prometheus, via a Puppet managed [[ https://wikitech.wikimedia.org/wiki/Data_Platform/Systems/Varnishkafka | varnish Kafka ]] producer called [[ https://wikitech.wikimedia.org/wiki/Performance.wikimedia.org/Runbook#statsv | statsv ]].
Product teams often choose to emit product related metrics via mw.track and statsv to take advantage of WMF's only supported publicly available dashboarding tool: [[ https://wikitech.wikimedia.org/wiki/Grafana | Grafana ]]. This metric data is stored in Prometheus, where it can also be used for automated alerting.
When teams to this, the data they emit is not available along side of all the 800+ datasets in the WMF [[ https://wikitech.wikimedia.org/wiki/Data_Platform/Data_Lake | Data Lake ]].
If we were able to query operational metrics via the WMF Data Platfrom, they would be available for joining with other data lake hosted datasets, and for building dashboards in Superset.
### Use Cases
- In T355837#10260100, @mpopov wrote:
> for Product Analytics it would be really helpful if the data was available in data lake and could be accessed/reported with Superset (which we know how to use).
>
> For various reasons, Web team decided to use statsv for instrumenting their small-scale experiments and thus ended up being a Grafana dashboard for the analyst to use (since we have no expertise on the team with that platform) – ref. T374965#10180679
- [[ https://grafana.wikimedia.org/d/bSDW0wJ7z/growth-team-product-kpis?orgId=1 | Growth team product KPIs ]]
- [[ https://grafana.wikimedia.org/d/d78efd40-141e-492e-b7cd-9c9482744034/growth-surfacing-structured-tasks?orgId=1 | structured task conversion metrics ]]
- [[ https://grafana.wikimedia.org/d/000000566/overview?orgId=1 | generic dashboard for Reading Web/mobile metrics ]]
- {T325790}
- [[ https://grafana.wikimedia.org/d/STSXVVdSk/mediamoderation-photodna-stats-legacy?forceLogin&orgId=1&refresh=5m&var-wiki=commonswiki&tab=alert | dashboard shows the status of scanning images on Wikimedia wikis using the MediaModeration extension ]]
### Possible solutions
#### Option 1: Produce OpenMetric/Prometheus compatible events via mw.track
As originally proposed at {T355837#10230113}, statsv.py was suggested to use the OpenMetric event as the canonical data, and then produce metrics to Prometheus. This is still an option, but in subsequent comments there were reasons this was not desirable.
Instead, a new `mw.trackSubscribe( 'stats.' )` JS handler function could do the following
- convert the `mw.track('stats.*', ...)` params into an OpenTelementry/Prometheus compatible JSON event, with data model something like [[ https://gerrit.wikimedia.org/r/c/schemas/event/primary/+/1072757/2/jsonschema/development/openmetric/current.yaml | this POC event schema ]]
- sendBeacon POST `/beacon/v2/event` (possibly using usual EventLogging JS API?)
- `/beacon/v2/event` -> eventgate-analytics-external -> kafka
- events will then automatically make it into a Hive table.
**Pros**
- We can eventually use the same site /beacon/v2/event endpoint for all externally submitted events, e.g. from EventLogging, removing the extra http connection to intake-analytics.wikimedia.org. {T263049}
- The prometheus data model is mostly [[ https://druid.apache.org/docs/latest/ingestion/schema-model/ | compatible with Druid ]], so we could easily consume this data for use in [[ https://turnilo.wikimedia.org | turnilo.wikimedia.org ]]
**Cons**
- Emits the data twice from browsers, once for `/beacon/statsv` (dogstatsd format) and once for `/beacon/v2/events` (JSON event format).
- Compatibility: Does not support HTML and CSS-based (non JavaScript) instrumentation. EventGate does not support HTTP GET. (NOTE: If GET support is a requirement, it could be added to EventGate).
#### Option 2: produce OpenTelemetry event only
The first con could be overcome if we were to rely on the OpenTelemetry event format as the only data emitted, and transform that in statsv to produce to Prometheus. This was the original proposal in {T355837#10230113}
**Pros**
- data is only emitted once from clients
- removes CDN (varnishkafka) logic
**Cons**
- Interferes with 'Tier 1 telemetry'
#### Option 3: `mw.track('metric.*', ...)` topic prefix
Another alternative would be to provide a separate `mw.track` topic prefix for non 'tier 1 telemetry' metrics, e.g. `mw.track('metric.*', ...)` Product teams could then use this mechanism instead of the 'stats.' topic prefix when they emit product metric telemetry data.
A service similar to statsv.py would consume these metric events and produce them to prometheus.
**Pros**
- data is only emitted once from clients
- existent usages of `mw.track('stats.*', ...)` would not be available in the Data Lake.
- A new service (if not in statsv.py) to maintain
#### Option 4: dogstatsd kafka topic event transformer
Originally proposed in {T355837#10416983}.
After T355837, `mw.track('stats.* ... )` ends up producing [[ https://docs.datadoghq.com/developers/dogstatsd/datagram_shell/?tab=metrics | dogstastd ]] formatted metrics to Kafka.
Something (stastv.py or another service) could consume and transform this metric data into an OpenMetric/Prometheus compatible event and produce it to an Event Platform stream(s). This stream(s) would then be automatically ingested into Data Lake table(s).
**Pros**
- metric data is only emitted once from browsers.
- Supports HTML and CSS-based instrumentation without EventGate modifications
**Cons**
- A new service to (if not in statsv.py) to maintain.
#### Option 5: Prometheus metric kafka topic event transformer
Originally proposed in {T390328#10854337}
Prometheus has a [[ https://prometheus.io/docs/prometheus/2.55/configuration/configuration/#remote_write | remote_write ]] configuration that will allow it to emit (samples?) of metrics elsewhere for long term storage. It has both HTTP (protobuf) and [[ https://github.com/Telefonica/prometheus-kafka-adapter?tab=readme-ov-file#json | Kafka ]] (JSON) support.
Once the data is in Kafka, something could consume and transform the metric data into an OpenMetric/Prometheus compatible event and produce it to an Event Platform stream(s). This stream(s) would then be automatically ingested into Data Lake table(s).
**Pros**
- Any Prometheus metric could be collected, not just those emitted via `mw.track`
**Cons**
- A new service to (if not in statsv.py) to maintain.
#### Option 6: {T347430}
This allows Presto to query metrics directly in Prometheus using SQL.
**Pros**
- Any Prometheus metric can be queried, not just those emitted via `mw.track`
- No extra services or data products to maintain
- As of 2025-06, already done!
**Cons**
- Analytical queries could overload prometheus
- Operational metrics are not captured as event data; they cannot be used in stream processing or with distributed Data Lake query engines (like Spark).
- Presto is great for working with small/medium-ish datasets. It will not be so good for joining large datasets (e.g. webrequest) together with operational metrics.