Page MenuHomePhabricator

Edits being flagged by review tool on enwiki aren't likely to be damaging
Closed, ResolvedPublic

Description

See https://en.wikipedia.org/wiki/Special:Contributions/Classicfilms. Many of this user's edits get flagged in the UI. However, ORES does not think they are damaging.

Here's some examples of revisions that get flagged:

And here's the ORES scores for them:

Note that our new models do better with addressing these edits:

I think that the problem here is that the sensitivity is set too high. I think that ORES' recommendation here might be a bit crazy (threshold @ 13.8% for 90% recall). We should probably bump this up to ~30% in the short term and get the new deployment of ORES out the door. Note that the new models don't look so crazy (threshold @ 17.3% for 90% recall). If were were to just deploy that right now, it looks like it would address the issues for User:Classicfilms

Event Timeline

Per model info I set the high (default) to 14% and the low 43%. Do you think we should set the threshold something else?

Change 306419 had a related patch set uploaded (by Ladsgroup):
Increase ORES threshold for enwiki

https://gerrit.wikimedia.org/r/306419

Change 306419 merged by jenkins-bot:
Make default ORES threshold soft (higher threshold)

https://gerrit.wikimedia.org/r/306419

Mentioned in SAL [2016-08-24T13:27:46Z] <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: Make default ORES threshold soft (higher threshold) T143738 (duration: 00m 59s)

Got it deployed. According to eval.php $wgOresDamagingDefault is properly set to soft but that does not reflect in the user preferences.

@Ladsgroup said a patch is needed in ORES extension, once tested on beta cluster we can get it deployed in the next SWAT window, either this european evening or tomorrow during eu SWAT window.

Change 306435 had a related patch set uploaded (by Ladsgroup):
Change ORES threshold to soft as default

https://gerrit.wikimedia.org/r/306435

Change 306435 merged by jenkins-bot:
Change ORES threshold to soft as default

https://gerrit.wikimedia.org/r/306435

Mentioned in SAL [2016-08-24T18:23:21Z] <thcipriani@tin> Synchronized wmf-config: SWAT: [[gerrit:306435|Change ORES threshold to soft as default (T143738)]] (duration: 01m 01s)