How many edits would Automoderator revert per day at different caution levels?
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Samwalton9-WMF
	Oct 13 2023, 2:33 PM

Description

As part of our model testing process I think it would be helpful to understand how many edits we can expect Automoderator to revert. This will be useful for the community to understand Automoderator's potential impact, and we can compare with anti-vandalism bots, which we researched in T341857.

This should incorporate the internal configurations we already want to consider, i.e. not taking action on:

Administrators
Bots
Self-reverts
New page creations

Question:

What would the average number of daily reverts be for Automoderator ...
- At the different caution levels we have in our test (0.97, 0.975, 0.98, 0.985, 0.99)
- On the English and Indonesian Wikipedias

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		KCVelaga_WMF	T348869 How many edits would Automoderator revert per day at different caution levels?
		Resolved		KCVelaga_WMF	T351057 Automoderator revert per day at different caution levels: top 150 Wikipedias

Event Timeline

Samwalton9-WMF created this task.Oct 13 2023, 2:33 PM

KCVelaga_WMF claimed this task.Oct 16 2023, 10:11 AM

KCVelaga_WMF changed the task status from Open to In Progress.Oct 16 2023, 11:12 AM

KCVelaga_WMF edited projects, added Product-Analytics (Kanban); removed Product-Analytics.

@Samwalton9-WMF

Average daily number of edits Automoderator would potentially revert per day at different thresholds (based on 2022 data)

wiki	t-0.99	t-0.985	t-0.98	t-0.975	t-0.97
English Wikipedia	46	100	165	235	308
Indonesian Wikipedia	1	2	4	6	8

Notebook

Samwalton9-WMF merged a task: T348556: Analyse breakdown of edits above different Revert Risk thresholds.Oct 17 2023, 11:57 AM

Thanks for this @KCVelaga_WMF! Could you repeat the analysis for the German and Japanese Wikipedias? These communities are also engaged with Automoderator so we decided to proactively provide data/analysis for them too. It would also provide us with a couple of wikis between these wiki sizes :)

Samwalton9-WMF added a project: Moderator-Tools-Team (Kanban).Oct 17 2023, 12:20 PM

Samwalton9-WMF moved this task from Ready to In Progress on the Moderator-Tools-Team (Kanban) board.

@Samwalton9-WMF Sure, here is the info for all four wikis

Wikipedia	t-0.99	t-0.985	t-0.98	t-0.975	t-0.97
German	2	4	7	10	13
English	46	100	165	235	308
Indonesian	1	2	4	6	8
Japanese	15	18	21	26	31

In T348869#9274933, @KCVelaga_WMF wrote:

@Samwalton9-WMF Sure, here is the info for all four wikis

Wikipedia t-0.99 t-0.985 t-0.98 t-0.975 t-0.97

German 2 4 7 10 13

English 46 100 165 235 308

Indonesian 1 2 4 6 8

Japanese 15 18 21 26 31

Huh - I'm surprised to see de.wiki with such low numbers here. I wonder why that is. FlaggedRevs maybe?

Samwalton9-WMF moved this task from In Progress to QA on the Moderator-Tools-Team (Kanban) board.Oct 24 2023, 9:18 AM

Samwalton9-WMF mentioned this in T349606: Generate Automoderator testing data for hr.wiki.Oct 24 2023, 11:57 AM

In T348869#9275250, @Samwalton9-WMF wrote:

Huh - I'm surprised to see de.wiki with such low numbers here. I wonder why that is. FlaggedRevs maybe?

I think all these numbers are relatively low, ~1 % of all reverted edits does not seem like much help. As I've commented here, it might be because of heavy use of Abuse Filter.
Efficient and timely blocking also helps. On hrwiki we've had an average of 200 (IP) blocks a month, since May 2022 they're mostly done by Abuse Filter, which lead to a huge reduction of vandalisms overall.

cwylo mentioned this in T349832: Generate Automoderator testing data for cs.wiki.Oct 26 2023, 3:29 PM

@Samwalton9-WMF: It looks like @KCVelaga_WMF completed the request so I am resolving this task. If this is incorrect, please re-open and let KC know what is missing. Otherwise if you have follow-up/additional requests expanding on this one, please create a new task and check with KC about its priority relative to existing planned work.

KCVelaga_WMF mentioned this in T351057: Automoderator revert per day at different caution levels: top 150 Wikipedias.Nov 13 2023, 7:37 AM

KCVelaga_WMF changed the status of subtask T351057: Automoderator revert per day at different caution levels: top 150 Wikipedias from Open to In Progress.Nov 13 2023, 7:40 AM

KCVelaga_WMF closed subtask T351057: Automoderator revert per day at different caution levels: top 150 Wikipedias as Resolved.Nov 13 2023, 8:32 AM

KCVelaga_WMF reopened subtask T351057: Automoderator revert per day at different caution levels: top 150 Wikipedias as Open.Nov 21 2023, 4:20 PM

KCVelaga_WMF closed subtask T351057: Automoderator revert per day at different caution levels: top 150 Wikipedias as Resolved.Nov 22 2023, 12:44 PM

Scardenasmolinar moved this task from QA to Done on the Moderator-Tools-Team (Kanban) board.Jun 11 2025, 11:11 PM