• Title of session: Inside Automoderator: Writing and deploying a new moderation extension
• Session description:

The Moderator-Tools-Team is currently developing a new extension - Automoderator. It uses the Revert Risk model via the LiftWing API to score new Wikipedia edits and revert them based on scoring criteria defined and set by community administrators.

We will use this session to give an overview of the project and discuss our technical approach, sharing and inviting input on our approach to:

• Writing a new extension for deployment in production
• Leveraging the LiftWing API to retrieve model scores
• The job queue we're building to track Automoderator's progress
• Handling complex revert scenarios
• Integrating #growthexperiments-communityconfiguration

If you're interested in Automoderator for your community we would also love to connect about how and when Automoderator will be ready to deploy on new wikis!

• Username for contact: @Samwalton9-WMF and @Scardenasmolinar
• Session duration (25 or 50 min): 25 min
• Session type (presentation, workshop, discussion, etc.): Brief presentation followed by discussion
• Language of session (English, Arabic, etc.): English
• Prerequisites (some Python, etc.): None
• Any other details to share?: No
• Interested? Add your username below:
  • @KCVelaga_WMF

Notes from session

Inside Automoderator: Writing and deploying a new moderation extension

Date and time:

Relevant links

• Phabricator task: https://phabricator.wikimedia.org/T361259
• Session slides: https://docs.google.com/presentation/d/1gsTbWjAs8Is5FrHEasBomzXfepQvHL_vHBQZOIUGGdI/edit?usp=sharing
• https://www.mediawiki.org/wiki/Moderator_Tools/Automoderator

Presenter

Sam Walton - Samwalton9 (WMF)
Susana Cardenas Molinar (WMF)

Participants

Notes

Sam - Product Manager at the Moderator Tools Team
Susana - Software Engineer at the Moderator Tools Team
What if we ould automatically find the bad edits and revert them to prevent the need for a person to manually deal with this.
Large wikis can have bots that deal with anti-vandalism, but this does not work for small wikis which often don't have them.
It uses the revert risk model, recently made by the foundation, to determine what edits should be reverted based on a threshold
Automoderator looks like a normal bot account that does the revert in the edit history to make it clear what the tool does and to make it easier to track the edits
The configuration for this tool will be done via the CommunityConfiguration extension allowing admins to do this configuration themselves making it easier for local decisions to change the threshold, change messages, and turn on/off the tool
Deployment plan: Next 2 weeks on the beta wiki, May for Indonesian Wikipedia, June for the Turksish Wikipedia, and beyond that we are not sure of a set plan.
Automoderator will check almost every edit made which means it might take a while longer to have the edit completes. the solution was to come up with a job queue to view the edit. Automoderator will not check reverts, new page creations, self-reverts, bot edits, edits to protected pages.
Using the JobQueue to analyse edits will mean that some reverts will happen a little after the edit which may cause a revision conflict (because someone may have edited the page since or made the revert).

Questions

Q: You mentioned that it doesn't edit protected pages. Does this include autoconfirmed pages or fully protected pages.
A: We have specifically given it the bot user group, so we tried to match it with the rights that it has.

Q: How does this work with FlaggedRevs? Does it revert the pending edit or reject it?
A: We recently tested this. The edit that does the revert will revert the pending edit and because the user has the bot group, it has the auto patrol right which means that the revert and the edit will be marked as accepted (because the revert removed that content).

Q: Is this specific to Wikipedia?
A:At the moment the model we are using is for Wikipedia but we have plans to expand to wikidata.

Q: Does this revert risk model support all languages or is it trained per wiki?
A: The generic revert risk model supports all languages for edits to the article namespace. There is a more complicated model that is language specific but it only supports 40 languages and is currently too complex to run.

Q: Do you think this could replace the bots on the exisitng wikis?
A: We will need to discuss with these wikis. Some of the wikis using the revert risk model, so it means that the data set is the same (Romanian wiki). However, for bots like ClueBot we would need to work with the community to discuss.

Q: In the mockup, you had the cautious vs less cautious sessions that defined the approximate edits that are being reverted. Would the number of reverted edits be automatically generated?
A: In the mockup the edits is static and making it change per wiki is on the roadmap.

Q: How can communities reach out to the team?
A: You can use the mediawiki page for the tool - https://www.mediawiki.org/wiki/Moderator_Tools/Automoderator

Q: How are you collecting data right now?
A: We are collecting data through the testing we did before and we have a big measurement plan listed at the mediawiki page ( https://www.mediawiki.org/wiki/Moderator_Tools/Automoderator )

Q: How might using the JobQueue affect the time it takes to revert on mass attacks / DDOS?
A: Using the job queue default of three attempts. We based it on ORES, which has proved itself through time and being deployed to large wikis