Page MenuHomePhabricator
Create Task
Maniphest T348860

Baseline: What is the median time for vandalism to be reverted?
Closed, ResolvedPublic
Actions

Description

Research question: What is the efficiency of Automoderator in countering vandalism on wikis?

Evaluation method: Extent to which Automoderator minimises the exposure of vandalised content to readers

Baseline
We will be comparing this value pre- and post- Automoderator deployment directly.

Question: What is the median time for vandalism to be reverted by any other user?

This analysis should be done individually for the top 10 Wikipedia projects by editing activity: en, es, ja, de, fr, ru, zh, it, pt, fa, plus our pilot id.wiki

Here we will try out our operational definitions:

Vandalism

  • Reverted within 12 hours
  • User edit count less 15 edits
  • Time since user's first edit is less than 48 hours
  • Reverted by a different editor
  • Edit is in the main namespace

Related Objects
Search...

Event Timeline

Samwalton9-WMF created this task.Oct 13 2023, 1:23 PM
Samwalton9-WMF assigned this task to KCVelaga_WMF.Oct 13 2023, 1:26 PM
Samwalton9-WMF renamed this task from What is the median time for vandalism to be reverted? to Baseline: What is the median time for vandalism to be reverted?.Oct 13 2023, 1:31 PM
Samwalton9-WMF added a parent task: T348864: Research question: Is automoderator effective in countering vandalism on wikis?.
KCVelaga_WMF edited projects, added Product-Analytics (Kanban); removed Product-Analytics.Nov 20 2023, 9:32 AM
KCVelaga_WMF changed the status of subtask T349083: Compare the revert risk scores with proposed criteria for vandalism from Open to In Progress.
Samwalton9-WMF closed subtask T349083: Compare the revert risk scores with proposed criteria for vandalism as Resolved.Dec 12 2023, 5:06 PM
Samwalton9-WMF updated the task description. (Show Details)Dec 13 2023, 5:40 PM
KCVelaga_WMF added a comment.EditedDec 18 2023, 4:26 AM

@Samwalton9-WMF

Median Time to Revert for Probable Vandalism

Wikipediasecondsminutes
German Wikipedia1863.1
English Wikipedia5699.48
Spanish Wikipedia601
Persian Wikipedia1983.31
French Wikipedia4357.25
Indonesian Wikipedia308551.42
Italian Wikipedia3585.97
Japanese Wikipedia113918.98
Portuguese Wikipedia102317.06
Russian Wikipedia77712.95
Chinese Wikipedia204334.06
Johannnes89 subscribed.Dec 20 2023, 9:48 PM
Samwalton9-WMF added a comment.Dec 21 2023, 10:00 AM
In T348860#9411971, @KCVelaga_WMF wrote:

@Samwalton9-WMF

Median Time to Revert for Probable Vandalism

Wikipediasecondsminutes
German Wikipedia1863.1
English Wikipedia5699.48
Spanish Wikipedia601
Persian Wikipedia1983.31
French Wikipedia4357.25
Indonesian Wikipedia308551.42
Italian Wikipedia3585.97
Japanese Wikipedia113918.98
Portuguese Wikipedia102317.06
Russian Wikipedia77712.95
Chinese Wikipedia204334.06

Thanks! This is perfect, and we'll revisit the numbers after Automoderator has been deployed.

Samwalton9-WMF added a project: Moderator-Tools-Team (Kanban).Dec 21 2023, 10:00 AM
Samwalton9-WMF moved this task from Ready to Done on the Moderator-Tools-Team (Kanban) board.
Pablo mentioned this in T346095: Investigate the tangible differences between the results from goodfaith vs revertrisk.Dec 21 2023, 1:22 PM
KCVelaga_WMF triaged this task as Medium priority.Jan 6 2024, 3:22 PM
KCVelaga_WMF moved this task from Next 2 weeks to Needs Review on the Product-Analytics (Kanban) board.
Samwalton9-WMF closed this task as Resolved.Jan 9 2024, 5:06 PM
KCVelaga_WMF mentioned this in T355789: Update Automoderator baselines based on 2023 data.Jan 24 2024, 1:06 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL