Page MenuHomePhabricator

Examine recent trend of decline in revert rate across projects
Open, NormalPublic

Description

With a dropping revert rate we can think of a few different causes, not mutually exclusive, with very different implications on our efforts:

  1. Quality of edits is getting better
  2. Quality hasn't changed, but oversight is getting worse-->leading to lower overall quality
  3. Quality hasn't changed, but oversight is getting more forgiving, leading to no change in overall quality

Questions around the scope and vector

  • is it project specific or universal?
  • is it correlated to edit increases?
  • it it correlated to anon/bot/user tenure status?

Most direct quality question

  • Can we look at average ORES scores of un-reverted edits over time? That might get us what we need to know. But there might be modeling artifacts that confound the results. We'd need to run it by Aaron @Halfak who created and maintains ORES.

Event Timeline

JKatzWMF created this task.Thu, Sep 19, 8:41 PM
JKatzWMF updated the task description. (Show Details)
cchen added a comment.Wed, Sep 25, 3:34 PM
  • Projects: Considering the YoY % decrease in revert rate and revert volumes, wikidata, enwiki, dewiki, ruwiki, zhwiki, are showing a big drop in revert rate for past few months.
  • Drops in revert rate are correlated to (non-bot) Edit Increase for some projects: The drop in recent revert rate is correlated to (non-bot) edit increases only in wikidata and commons. For some other projects (enwiki, dewiki, ruwiki, zhwiki, frwiki, itwiki). the increases in (non-bot) edits are small over time, and the drop in reverts result in the drop in revert rate.
  • Drops in revert rate/reverts are not correlated to number of users (new & returning) who make reverts: By looking at the number of users who make reverts overtime by tenue and by project, the numbers didn't change much over YoY.

Thank you for the update. Do you have a link to your analysis work?

Wait a moment. Is this just due to summer time? We should expect to see an increase in vandalism in September and a drop in May. This is common. It appears and seems to happen every year. I think it is related to students in the west going back to school/leaving on summer vacation. There's a lot of vandalism of the form "<school official of some sort> is <derogatory term>" that just doesn't happen in the summer.

cchen added a comment.Tue, Oct 1, 5:27 PM

The link is here.

The drop in revert rate is compare the revert rate this year to the same time period last year. It might not be due to summer time, but I will check how the data looks like for the back to school month.

kzimmerman assigned this task to cchen.Mon, Oct 7, 5:12 PM
kzimmerman triaged this task as Normal priority.
kzimmerman moved this task from Triage to Doing on the Product-Analytics board.