Page MenuHomePhabricator

Production monitoring for TechWishes scraper
Open, Needs TriagePublic

Description

The Technical Wishes scraper publishes Prometheus metrics to be monitored on the dashboard.

The scraper runs intermittently, so it should push metrics to an existing push gateway (pushgateway source) to cache metrics across jobs.

(Deprecated) Patch 1037437 enables Prometheus to scan the scraper's metrics endpoint. Abandoning in favor of the push gateway.

  • Write a push method compatible with the Elixir prometheus client
  • Publish /metrics to pushgateway 1x/minute
  • Use job name "wmde-content-summary"
  • Deploy and verify metrics.
  • Document operational metrics in metrics.md, in a section following analytic metrics.
  • Connect logging to logstash. Structured logging for eg. erroring page_ids.

Production gateway: http://prometheus-pushgateway.discovery.wmnet/

Code to review:

Logging to logstash should be done either by running the job in a container which forwards its stdout and stderr, or by logging to rsyslogd. See https://wikitech.wikimedia.org/wiki/Logstash/Interface .

Details

Related Changes in Gerrit:

Event Timeline

Change #1207174 had a related patch set uploaded (by Awight; author: Awight):

[operations/puppet@production] Monitoring for WMDE dumps scraper

https://gerrit.wikimedia.org/r/1207174

awight updated the task description. (Show Details)

Waiting on external review and discussion. The only question is whether it's okay for the job's Prometheus reporting endpoint to be unavailable when the job is not running.

awight renamed this task from Temporarily reenable production monitoring for the scraper to Production monitoring for TechWishes scraper.Dec 3 2025, 11:28 AM
awight updated the task description. (Show Details)
awight updated the task description. (Show Details)
awight added a subscriber: herron.

Change #1207174 abandoned by Awight:

[operations/puppet@production] Monitoring for WMDE dumps scraper

Reason:

Prometheus export will be done by the app using push gateway

https://gerrit.wikimedia.org/r/1207174

awight updated the task description. (Show Details)
awight updated the task description. (Show Details)
awight updated the task description. (Show Details)
awight removed awight as the assignee of this task.Dec 8 2025, 4:34 PM
awight updated the task description. (Show Details)

Remaining work is to smoke-test on production, and document.