OKR 5.2
User Story
"As a third-party reuser of wikimedia data, I want to know that the quality of the revision I am receiving in realtime is good,
so that I can better prioritize this new revision in my systems, whether of ingesting and surfacing to my audience or for training my models."
Why
Our customers often ask for a score. While a score may not be the best method, or the end all, be all of accurately assessing quality of articles across wikipedia, it may be a perfect funcional placeholder while we build to a better end in the longer-term.
What
Research has a built and tested a model for judging the quality of a revision in an article, which would need to be put into production via Liftwing and the ML team.
Acceptance criteria
- Engineering team determines that the model is functional in a realtime environment with goal of understanding if we should add a dependency in our APIs
To Do
- Test existing model
- Make determination of its offering as far as latency, quality of results and general offering. Could we add it to realtime? or only on demand?
- Liase with ML team on results
- Document decisions in One Pager with Francisco
An explanation of the score the model delivers —
Related ticket on Research quarterly report of model quality