We need a service that lives in toolforge that shows how is our support for the community. For example it shows me (live) stats about wikis that have different models enabled for them including some stats about their labeling campaigns and its progress over time. This sounds like fun.
Agreed. This is related to T140364: Model information UI (graphs and statistics). We should have a UI that is rendered based on queries to ORES and queries to Wikilabels. We can worry about rendering detailed model_information separately from rendering support-status, but I think the same UI should probably do both.
I'm just going to put this here:
select date_trunc('day', label.timestamp) "day", count(*) from label inner join workset_task on label.task_id = workset_task.task_id inner join workset on workset_task.workset_id = workset.id where campaign_id = 71 group by 1 order by 1;
This type of query can show progress of labeling over time. It's useful when we are building the community monitor.
What do you think about including some notion of language support? E.g. we could query github to see if there's a matching language folder in revscoring -- or just import revscoring in python and look for it.
I cleaned up the repo, cleaned up the home page, activated travis (it only checks for flake8, that's enough for now). Put an updater.sh in it to update itself and the data and put it in cron of the tool to run every three hours. Right now, I think we have a good enough of an MVP to call this done and put any other improvements to that tool in other dedicated phabricator tasks. Also this needs announcement and update of the documentation in mediawiki.org
It's hard to comment in GH, so I'll put some notes here:
- https://github.com/wiki-ai/ores-support-checklist/blob/master/updater.sh#L1 Hardcoded paths are bad news.
- https://github.com/wiki-ai/ores-support-checklist/blob/master/data_updater.py#L12 You could use halfak/deep-merge
- https://github.com/wiki-ai/ores-support-checklist/blob/master/data_updater.py#L19 Should clean up the hardcoded URL.
- https://github.com/wiki-ai/ores-support-checklist/blob/master/app.py#L36 Slightly more clear to call it JSON_PATH
- https://github.com/wiki-ai/ores-support-checklist/blob/master/templates/index.html I love the template! Two things to follow up on, * the "if" statements for each cell should have linebreaks, and * ampersands must be escaped as entities, even in literal URLs.
- data.json should have linebreaks. Python will output pretty JSON with json.dumps(data, indent=4)