Page MenuHomePhabricator

Monitoring for GrowthExperiments link recommendation task pool
Closed, ResolvedPublic

Description

GrowthExperiments' Add Link feature relies on a pool of tasks fetched from a recommendation service, vetted, and added to the database and search index by a cron job. This pool could run dry if the task generation process cannot keep up with the rate at which tasks are consumed, or when task generation errors out. We should have some monitoring and maybe low-urgency alerts in place for noticing that in time.

For monitoring the service itself, see T276769: Create Grafana dashboard for link recommendation service and document it on wikitech.

Event Timeline

Change 674579 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Monitoring: Add endpoint for metadata about suggested edit task pool

https://gerrit.wikimedia.org/r/674579

Change 675085 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Add Special page for showing newcomer task pool sizes

https://gerrit.wikimedia.org/r/675085

Change 674579 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Monitoring: Add endpoint for metadata about suggested edit task pool

https://gerrit.wikimedia.org/r/674579

Change 675085 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Add Special page for showing newcomer task pool sizes

https://gerrit.wikimedia.org/r/675085

We have Special:NewcomerTasksInfo, Grafana integration, and an API endpoint. The toolforge site that consumes data from the API endpoint will happen in T249987: Scale: GrowthExperiments wiki monitoring dashboard so I think we could mark this task as resolved.