The Technical Wishes scraper publishes Prometheus metrics to be monitored on the dashboard.
The scraper runs intermittently, so it should push metrics to an existing push gateway (pushgateway source) to cache metrics across jobs.
(Deprecated) Patch 1037437 enables Prometheus to scan the scraper's metrics endpoint. Abandoning in favor of the push gateway.
- Write a push method compatible with the Elixir prometheus client
- Publish /metrics to pushgateway 1x/minute
- Use job name "wmde-content-summary"
- Deploy and verify metrics.
- Document operational metrics in metrics.md, in a section following analytic metrics.
- Connect logging to logstash. Structured logging for eg. erroring page_ids.
Production gateway: http://prometheus-pushgateway.discovery.wmnet/
Code to review:
Logging to logstash should be done either by running the job in a container which forwards its stdout and stderr, or by logging to rsyslogd. See https://wikitech.wikimedia.org/wiki/Logstash/Interface .