Page MenuHomePhabricator

Save ORES-guided edit-review decision data to improve accuracy of ORES models
Closed, DuplicatePublic

Description

For a later phase of ERI, create system for continual tuning of ORES scores.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 9 2016, 12:44 AM

What is "ORES decision data"?

jmatazzoni renamed this task from Save ORES decision data to improve accuracy of models to Save ORES-guided edit-review decision data to improve accuracy of ORES models.Sep 9 2016, 12:46 AM

@catrolpe, I edited the task title. Does that help?

Yes, that helps, thanks.

IIRC @Halfak said something about this data not being directly suitable for training (because it was influenced by ORES's decision, so there's a feedback loop), but he also said that storing false positives would be useful.

Halfak added a comment.Sep 9 2016, 2:02 PM

Indeed. Storing false positives helps us notice trends and prioritize new work to make the models more effective.

Halfak added a comment.Sep 9 2016, 2:03 PM

See https://www.wikidata.org/wiki/Wikidata:ORES/Report_mistakes for an example of how we usually work with misclassification reports.