Hello! Just letting you know that you can schedule this session now for the hackathon. Instructions and scheduling can be done here:
https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2017/Program#How_to_schedule_breakout_sessions

I agreed to join Zache for the session. We'll probably spend 5 minutes recapping what FlaggedRevs is and giving a quick demonstration, for those who don't know or never tried its interface directly.

It is today 19-20 in room Wiaschtl

"for breakout sessions: 25 seats around tables in U-shape, fixed projector and screen, flip chart", description is incorrect and there is no tables in Wiaschtl. Most likely this means that presentation is short and we are moving to somewhere we can talk, eat pitsa and beer.

We also recapped T121995: Switch FlaggedRevs on Hungarian Wikipedia to a "flagged protection" mode a bit (the task could use some immediate action) and mentioned T163107: Flagged revs Special:ValidationStatistics update doesn't work. Some of the actionables involve ORES outreach on the wikis where the FlaggedRevs backlog needs help.
Is https://etherpad.wikimedia.org/p/flaggedhack17 ready to be copied on a wiki page?

Notes copied from: https://etherpad.wikimedia.org/p/flaggedhack17

Documentation

https://meta.wikimedia.org/wiki/Flagged_Revisions - needs improvement, Nemo and Zache have been working on it.
https://translatewiki.net/wiki/Special:MessageGroupStats/ext-flaggedrevs-0-all has statistics on translations (but includes API help messages as of now)
Stats http://tools.wmflabs.org/fiwiki-tools/fr_stats_example.html?db=fiwiki_p,huwiki_p,dewiki_p,plwiki_p&frs_key=pendingLag-average

Discussion

FR was primarily invented for de.wiki (German Wiipedia). There was very few tools to fight vandalism. Now we have abusefilter. We want to promote this, because easier to configure directly by the wiki admins, and react to new patterns.

Dereckson: FlaggedRevision is an old solution enabled on certain wikis before AbuseFilters or more modern tools were avaliable. With AF, FR are not relevant anymore. Nemo: at least for smaller wikis which are struggling with

Zache: on Finnish wikipedia, we are using AF quite extensively. From a wiki POV, setting up AF requires a lot of initial work, and technical expertise. In contrast, FR is quite straightforward to use and to explain.

Nemo: A Meta wikipage exists (https://meta.wikimedia.org/wiki/Flagged_Revisions), where the experience about FR is collected. Some of the sections a a bit opinionated, but it is okay. There is dozens of wikis where FR is enabled and we know nothing about them. Some of them don't respect the basic requirmeents, like translate the interface

Nemo: translatewiki, at some point all translations units were made optional because they are technical gibberish.
Dereckson / nemo : not a lot of languages have these messages translated, including languages used on wiki projects where the extension is deployed

https://tools.wmflabs.org/fiwiki-tools/fr_stats_example.html?db=fiwiki_p,huwiki_p,dewiki_p,plwiki_p&frs_key=pendingLag-average
A nw tool to show revision acceptance lag evolution. Y axis: seconds
If we add Uk or Ru we see growing backlogs.
On sq.wikipedia, we've a growing growing growing lag time. 70000000 seconds = 810 days

Edit a thon: are we sure edits are live?

CALL OF ACTION: watch stats on projects and raise alarm / check what happens when nobody / few people check revisions

Regular Arwiki discussions and phab tasks about removing the extension, but contentious.

Interface

Some wikis don't care about reviewing everything, because it's just extra meta-info - all readers see the latest edit. But the extra-interface creates some clutter. [?]
Some testing of the interface - e.g. Selenium, might be helpful. - Nemo will file a phab task to request QA input.
Visual editor's warning flyout menu is distracting, in comparison to the wikitext editor's mesage above the text-area.

Huwiki

Before implementing, we had govt issues around protected data, e.g. race, religion, political affiliation - if a person [requests? doesn't permit?] it's strictly forbidden to publish. Sometimes false information got into BLPs and the state ministry got reports of these problems. Data Protection and Minority Protection agencies. Long investigation ensued. Edits that were reverted minutes later, were still a problem. After FR we could say "this edit never became public".
Some edits never get checked in recent changes, but in FR it is impossible to miss anything.
Some edits should never be shown to the public. But it is quite tiring for the ~30 people who triage this backlog. There are regular campaigns to reduce the backlog.
We get a lot of email from readers and beginner editors about their contributions not appearing in a reasonable amount of time.
The entire point of wiki is that your edit is live immediately, if it isn't then contributors are confused and demotivated because the process is unclear and dissatisfying.
To fix that, there was some initiative on hu. to amend settings, so everybody can see latest version - most editors supported that we should show the latest version. - but there is a unique configuration at huwiki that [?] makes it easy to see if an edit is positive [?] https://phabricator.wikimedia.org/T121995 Some editors were just mass-approving all edits in order to reduce the backlog. Some not-so-easy cases were getting older and older (many weeks).
The number of active editors dropped off significantly around the time FR was implemented.
Unreviewed and Reviewed levels have a potentially unclear meaning.
We would like to have a 3rd level that shows "Was reviewed by an expert".

Data

Nemo: 20 wikis are using wgflaggedrevsoverride=false - IIUC that's the config that shows the most recent edit. Out of ~43 wikis that use the extension.

Other solutions

1. Replacing with ORES. A volunteer project, now supported by the Collaboration team, has released these last 12 months the Enhanced Edit Review system on many wikis as a beta feature - that may be a good alternative: https://www.mediawiki.org/wiki/Special:MyLanguage/Edit_Review_Improvements/New_filters_for_edit_review. Some filters on some wikis are based on ORES predictions. It is not solving everything, but it may be combined with AF.

2. SupplementingIntegrating with ORES. Add ORES information into the review queue to help them target good edits that are easier to review.

3. AbuseFilter is another alternative. How to get local users to test new filters?

4. FR: Getting the backend working, for instance fixing Special:Validationstatistics, so that users can review more productively, would be helpful.

Fiwiki has the pending changes link in the mediawiki:sidebar, which helps a lot.

Get some research done about the effects of showing the latest revision instead of the stable version on wikis where only the stable version is supposed to be shown: for instance compare two articles on hu.m.wikipedia.org, one of which has a pending revision and one not. Does showing the "wrong" revision on the mobile website produce some identifiable "harm"?

Immediate action item: advertise that ORES provides, on Special:RecentChanges, an automatic list of probably-bad faith edits which need FlaggedRevs review, and ask them whether they'd like to try and use this to speed up the review (or at least the urgent reviews). Only avaliable on wikis where the ORES damaging and good faith training has been done and integrated (see https://www.mediawiki.org/wiki/ORES#Support_table), in short Polish Wikipedia only, so it has a setup cost.

Perhaps ORES can flag when an edit is "bad" and put it in the FR queue? But you can't do this on wikis where reviewing a revision alters whether the latest revision is shown.

ORES and AF can't replace FR for projects where showing last revision is a liability in itself. That is it does not block contentious revisions by default, it can only warn about contentious revisions. It can although automate some actions in FR, like protecting likely bad revisions or publishing likely good revisions, thereby lowering the strain on those doing the triage. (T165848: Decrease FlaggedRevs backlog by using ORES predictions models)

The privacy issues should not be dismissed, as those will be a problem in the future. It is a somewhat political issue on Wikipedia, and a lot of users believe they can write whatever they want, but that is not the case in Europe. We do need better mechanisms to handle vandalisms, especially on articles about living people. If an article is a biography about a living person, the FR and other supporting systems should error on the cautious side and probably default to "always review" or at least nearly always review.

Contentious revisions should also be removed from the history of the article, see T164494: Reduce history to real contributors, that is remove vandalism

Follow up actions.
There is a bot based stabilization now which does a 24 hour stabilization to the page after a problematic edit. The bot follows the EventStrean and stabilization is triggered if the score of the edit is high enough. In fiwiki the score is based on how many rules like ORES, Abuse filter, graylisted ip range etc edit triggers and least two rules need to be triggered for stabilization.

Currently there is a 5 min delay between edit and stabilization so that vandal fighters can revert the edit before the stabilization because currently stabilization will add a null edit and after the null edit the rollback link tries to revert the null edit and not the edit which triggered the stabilization. There is patch T169457 which will fix this but it waits the approval.

Code:

• https://github.com/4shadoww/stabilizerbot

Bot in action

• https://fi.wikipedia.org/wiki/Toiminnot:Muokkaukset/VakauttajaBot

Bot configuration

• https://fi.wikipedia.org/wiki/user:VakauttajaBot/config.json