On wikis where ORES predictions and FlaggedRevs are both enabled, consider to have good faith edits to be automatically marked as patrolled. This would decrease FlaggedRevs backlog, very high on some wikis.
|Open||None||T165848 Decrease FlaggedRevs backlog by using ORES predictions models|
|Open||awight||T166235 Flagged revs approve model to fiwiki|
- Mentioned In
- T163197: Flagged revisions discussion: Current state and future plans
- Mentioned Here
- T72457: Set a limit on pages with forced review when there are too few active reviewers
T72452: Mark edit as pending review for all users from the same IP-address/range
T132901: Use revision scoring to trigger flagged protection
T163197: Flagged revisions discussion: Current state and future plans
In wikis where the backlog is very long the backlog will need to be reseted somehow before it will work because one problem is that the there is old pending change and new edits will be pending too because that even if they were normally autoreviewed. Basic case is like this where there is trivial change blocks the automatic reviewing in newer edits.
Basic concept is solid though and it you could be using also the historical data too. Eg edit is automatically patrolled IF the edit is goodfaith AND if the last N edits from that user is good faith too.
If needed one safety check could be that if there is Abusefilter hits/added tags then do not automatically review it.
I tested the idea little bit. It would work with registered users pretty nicely, but least in fiwiki scores for ip-editors scores are generally lower than what they should be and it would need some help with that. In pl wiki if the scores are reliable it would work well.
Average value of scorer goodfaith (true) counted from last 5000 edits in selected wikipedias
- all = all editors
- users = all registered editors
- autoreviewed users = edits which were automatically approved with FlaggedFevs
- non_autoreviewed = edits from registered editors which were not approved automatically
- ip = edits from ip users
lang all users autoreviewed non_autoreviewed ip ru 0.831 0.951 0.996 0.879 0.354 pt 0.751 0.904 0 0 0.495 pl 0.962 0.972 0.974 0.964 0.889 he 0.978 0.995 0 0 0.807 fi 0.818 0.943 0.979 0.818 0.395 fa 0.858 0.926 0 0 0.392 en 0.831 0.919 0 0 0.56 cs 0.862 0.935 0 0 0.341
I'm not familiar with those processes.
Like I understand them, the best would be to have something done immediately, to avoid backlog or usefulness log recording (flagged, and then unflagged).
For some languages a vandalized revision should not be shown if it is a biography, that is the article is in a tagged category, which imply the ORES prediction must have an effect immediately. ORES should then hide the revision pending further review.
Here is a proof-of-concept style pywikibot script for reviewing the Flagged Revs backlog which also works in other Wikipedias apart from the Finnish Wikipedia.
Rules used for approvals are:
- approve if the edit was made by a user who was autoreviewed/patrolled
- approve if the edit was made by a bot
- approve if the edit was a revert or reverted
- approve if the edit was an interviwiki change
- approve if the edit was well scored using ORES
What would be the next wiki to try? One of these, which have FlaggedRevsOverride true? https://tools.wmflabs.org/fiwiki-tools/fr_stats_example.html?db=fiwiki_p,huwiki_p,dewiki_p,plwiki_p&frs_key=pendingLag-average
Among them, idwiki and arwiki seem to have basic ORES support.
My request was posted by the local users in their village pumps, thanks!
My guess is that the best use cases would plwiki and ruwiki. Both have advanced ORES support enabled and strong technical community for running and modifying the bot if they want to do so. Next after that would be huwiki which is currently doing ORES advanced support labeling. Ukwiki would be also interesting choice if Bunyk who was at the hackathon is interested for participating.
ORES advanced support is important because we can't really pull out the reviews from thin air and goodfaith scorer seems to be a lot better predictor than the reverted scorer.
Also a project page at the meta would be useful if we want to get different wikiprojects to participate.