Page MenuHomePhabricator

Save ORES-guided edit-review decision data to improve accuracy of ORES models
Closed, DuplicatePublic

Description

For a later phase of ERI, create system for continual tuning of ORES scores.

Event Timeline

What is "ORES decision data"?

jmatazzoni renamed this task from Save ORES decision data to improve accuracy of models to Save ORES-guided edit-review decision data to improve accuracy of ORES models.Sep 9 2016, 12:46 AM

@catrolpe, I edited the task title. Does that help?

Yes, that helps, thanks.

IIRC @Halfak said something about this data not being directly suitable for training (because it was influenced by ORES's decision, so there's a feedback loop), but he also said that storing false positives would be useful.

Indeed. Storing false positives helps us notice trends and prioritize new work to make the models more effective.

See https://www.wikidata.org/wiki/Wikidata:ORES/Report_mistakes for an example of how we usually work with misclassification reports.