What will it take to start using the reverted models in the ORES review tool? What considerations should be in place for using this sub-par vandalism prediction strategy?
This task is done when we have filed phab tasks for all the things that need to be done in order to enable the reverted model on at least one wiki.
@jmatazzoni, if we do something like this, that would extend the ORES review tool support to 24 wikis (up from the current 8). But this model is less good. We'd like to use this as a carrot to encourage people to work on the Wikilabels campaigns (to enable the damaging and goodfaith models).
Given that we are not responsible for the UI, I leave this decision to @jmatazzoni if he thinks it's good, I move forward with this.
Hi @Ladsgroup. I'm not exactly sure what implementation means here. Are you referring to adding the Reverted model to Recent Changes? If that is a similar but inferior model to the Damaging model, I don't understand what he point would be. Can you elaborate?
Also, I'm not with the Collab Team any more, so @MMiller_WMF, the new PM there, would be the person to discuss this with. Good luck.
Hey @Ladsgroup -- I'm also interested in the answers to Joe's questions, but I can say that I don't think the team plans any additional work on Recent Changes in the foreseeable future, since we're focusing on the work you're doing with Stephane on AfC, and then we'll be moving on to our core growth work.
@jmatazzoni: Sorry for bothering. I didn't know that.
So reverted model is basically damaging model that is build based on auto labeled data which means accuracy of the model is lower but it can be useful in case when the wiki doesn't have human labeled data and it's a way to encourage people to label data by deploying a model that is not as good as it can be and asking them to label edits to improve its predictions.
Thanks, @Ladsgroup. That makes sense. So like I was saying before, we aren't currently planning on expanding on the Recent Changes feed work. But we can keep this idea as a reference in case we end up wanting to add models to wikis that don't have the damaging model. Is there a list of wikis for which there exists a reverted model? Or can it easily be built for any wiki?