Page MenuHomePhabricator

Build ORES support checklist
Closed, ResolvedPublic

Description

We need a service that lives in toolforge that shows how is our support for the community. For example it shows me (live) stats about wikis that have different models enabled for them including some stats about their labeling campaigns and its progress over time. This sounds like fun.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Agreed. This is related to T140364: Model information UI (graphs and statistics). We should have a UI that is rendered based on queries to ORES and queries to Wikilabels. We can worry about rendering detailed model_information separately from rendering support-status, but I think the same UI should probably do both.

I'm just going to put this here:

select date_trunc('day', label.timestamp) "day", count(*) from label inner join workset_task on label.task_id = workset_task.task_id inner join workset on workset_task.workset_id = workset.id where campaign_id = 71 group by 1 order by 1;

This type of query can show progress of labeling over time. It's useful when we are building the community monitor.

Ladsgroup renamed this task from Build Scoring platform community monitor to Build ORES support checklist.Mar 26 2018, 3:49 PM

https://tools.wmflabs.org/ores-support-checklist/

Things to do:
1- link to the source code
2- make the table sortable
3- add cronjob to update the data every three hours
4- automatic update of the tool (using cron)
5- clean up the source code: https://github.com/wiki-ai/ores-support-checklist

What do you think about including some notion of language support? E.g. we could query github to see if there's a matching language folder in revscoring -- or just import revscoring in python and look for it.

I cleaned up the repo, cleaned up the home page, activated travis (it only checks for flake8, that's enough for now). Put an updater.sh in it to update itself and the data and put it in cron of the tool to run every three hours. Right now, I think we have a good enough of an MVP to call this done and put any other improvements to that tool in other dedicated phabricator tasks. Also this needs announcement and update of the documentation in mediawiki.org

It's hard to comment in GH, so I'll put some notes here:

The repo is great as it stands, my comments above can happen in follow-up work if at all.

What do you think about including some notion of language support? E.g. we could query github to see if there's a matching language folder in revscoring -- or just import revscoring in python and look for it.

This would be awesome, worth a followup task.

awight mentioned this in Unknown Object (Phame Post).May 2 2018, 6:41 PM
awight mentioned this in Unknown Object (Phame Post).
awight mentioned this in Unknown Object (Phame Post).