There are two (not necessarily exclusive) ways that come to mind, a per-wiki special page and a Toolforge tool showing data for all wikis in a single view. It seems nice to be able to have an overview of all wikis so maybe Toolforge is the better option.

I don't think there's a nice way to find out which wikis have suggested edits enabled, but we could fetch the growthexperiments dblist from some git mirror, and see if the API endpoint exists for each.
After that, it's just 6 API queries per site, or a few hundred if we want to get the full (task type X topic) matrix. That's a lot so some form of caching would have to be involved.

Potentially we could leverage some initial exploration done for a similar project, the Community Health Metrics Kit. For this project, @alexhollender also created a prototype of how information could be imported into a dashboard in Google spreadsheet as an option.

Dashboard sheet eg

Compare projects eg

Compare metrics eg

Idea from retro today (@kostajh @Catrope): perhaps the first version of a dashboard could allow users to input templates and see how many tasks will be available per topic with those templates. Then they could export that as a start to their on-wiki configuration for newcomer tasks. Naturally, we would want this for wikis that do not have our features enabled yet.

I'd like to propose a barebones version of this so that we can get T276795: Monitoring for GrowthExperiments link recommendation task pool done sooner rather than later:

• create a new PHP project on toolforge, growthmonitoring or something like that
• the project has a config file that we can add wiki IDs we're interested in monitoring
• add an API endpoint (with 24 hour caching) to GrowthExperiments that
  • iterates over ORES topics from https://www.mediawiki.org/wiki/MediaWiki:NewcomerTopicsOres.json
    • iterates over task types
      • performs a search for each task type / ORES topic to get a task count
• the growthmonitoring project has a script that runs each day, calls the API endpoint for each wiki we are monitoring, and writes the totals into a per-wiki JSON file with a new row for each day
• we implement some basic chart / graph displays to show current status and trends with HTML + JS + CSS

EDIT: this is pretty similar to T249987#6048491, sorry I just scrolled back and saw that.

Change 674579 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Monitoring: Add endpoint for metadata about suggested edit task pool

https://gerrit.wikimedia.org/r/674579

Another option would be pushing the counts into statsd. AIUI pushing a huge number of metrics to statsd is not nice, but we could 1) use it as a temporary solution while there are only 4 target wikis, 2) instead of a separate metric for each of the 64 ORES topics, have some sort of buckets (topics with 500+ tasks, topics with 400-499 tasks etc). Instead of an API we'd have to use a cronjob, or maybe push the numbers whenever someone actually searches for them (although that is more fragile). Graphing/dashboarding options would be more constrained but less work (via Grafana), and we'd get alerts for free.

Prometheus supports multi-dimensional data points so maybe that could be a longer term option? I don't know anything about Prometheus though so that's just a blind guess.

In T249987#6941193, @Tgr wrote:

Another option would be pushing the counts into statsd. AIUI pushing a huge number of metrics to statsd is not nice, but we could 1) use it as a temporary solution while there are only 4 target wikis, 2) instead of a separate metric for each of the 64 ORES topics, have some sort of buckets (topics with 500+ tasks, topics with 400-499 tasks etc). Instead of an API we'd have to use a cronjob, or maybe push the numbers whenever someone actually searches for them (although that is more fragile). Graphing/dashboarding options would be more constrained but less work (via Grafana), and we'd get alerts for free.

Prometheus supports multi-dimensional data points so maybe that could be a longer term option? I don't know anything about Prometheus though so that's just a blind guess.

We discussed this yesterday. @Tgr will pursue the statsd approach and in parallel we'll also seek to expose some stats via an API endpoint that could be consumed by a HTML/JS app that produces some charts. I'll make subtasks.

Number of tasks available for each task type, for each topic

I'm making a subtask for this one.

Change 674579 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Monitoring: Add endpoint for metadata about suggested edit task pool

https://gerrit.wikimedia.org/r/674579

Not currently working on this.

Which features are enabled or disabled

Split out into T297661: Scale: GrowthExperiments configuration dashboard.