We have done lots of analysis on Suggested Edits but it has been a while since I looked at revert rates. I can work on a more comprehensive overview soon but off the bat, for Android app edits made on Wikidata data (descriptions) for the date range 2021-10-01 - 2021-11-30, the edit type `description change' (event_comment = 'wbsetdescription-set') has the most egregious revert rate. This is a bigger conversation obviously, but thought this was an interesting data point. Adding @Johan (and @JTannerWMF) as I told him I would get some data pulled together soon.

2021-11

2021-10

@SNowick_WMF I think you have a systematic bias here: changes to a description are more likely to be reverted by editors watching the page, compared with additions of new descriptions to unwatched pages. You could perhaps control that bias by looking at the number of people watching the article, and/or if the revert was done by an editor that had previously edited the page. But the really low revert rate for translations is interesting, and worth exploring more!

Remember also, that a "bad" Short description is often not reverted -- just re-edited. Do the stats just test for the "revert" tag?

@GhostInTheMachine Absolutely, we're well aware of this.

Hi @GhostInTheMachine and @Mike_Peel , thanks for filing this ticket. I'm the Product Manager for the Android app and was out of office Thursday and Friday. I'm ramping back up and a little under the weather but will get back to you by the end of the week with some next steps. I hope to partner with you on this challenge. Thanks again for filing!

Thanks again for your engagement on this @GhostInTheMachine and @Mike_Peel.

The next steps for our team is to investigate the revert rates of short descriptions from Android relative to other platforms and to see if we can run a script to determine how many of the short descriptions were rewritten. Additionally we want to investigate the quality of short descriptions by tenure. We hope to also engage medium sized wikis to get their point of view so that we understand if there is a difference in preference for quality vs. having content at all as a wiki matures.

Based on the investigation described above, we will determine what tenure constraints and better guidance to put in place to improve this experience. Before we make any decisions, we will update this task so we can collaborate.

Is there anything in this approach you'd like us to reconsider or add?

Also, just to give a clear timeline, we have a good number of people on vacation starting the 20th thru January 3rd, so I can't commit to this being fixed in the next few weeks, but I will commit to providing an update in January.

Hi Everyone! Just wanted to cross post here what we shared on EN Wikipedia:

So we looked into this and wrote an update here: mw:Wikimedia Apps/Team/Android#Special Update unrelated to Collaboration tools (January 2022):. We'll continue keeping an eye on it while we keep working on the communication tools in the app, and then return to the usefulness of suggested edits and how we can make them better in July. We appreciate the issue being raised – digging into the numbers, we found things that surprised at least me, e.g. I had thought that we'd have a significant number of bad edits being overwritten and not turning up in the reverts statistics, but this turned out not to be case. Do let us know about any issues, as always.

We did as suggested and ran a script to evaluate rewrites and it came back with only 2% of edits from Suggested edits being rewritten in a 30 day period. We were quite surprised and even went back to manually check, but it was an accurate number. You can read the full details of our analysis on MediaWiki. We posted there so that folks from other language wikis could read about it as well. We will keep progressing on the communications work and return to this in July. Thanks for raising it again.

@JTannerWMF It might be interesting to look at the time between the edit and the rewriting/revert, rather than just looking within 30 days. I suspect there will be a long tail to this distribution that will extend much beyond 30 days (e.g., I've reverted cases after several months) - but I'm not sure how significant that will be.

Is there anything we can do in the short term to prevent brand-new editors from adding bad short descriptions? Here's an example from today of an editor's first edit being a bad SD: https://en.wikipedia.org/wiki/Special:Contributions/197.231.239.124

And another from today: https://en.wikipedia.org/w/index.php?title=Lee_National_Denim_Day&type=revision&diff=1072077741&oldid=1047745791