Provide regular cross-wiki reports on flagged revisions status
Open, LowestPublic

Description

Same as bug 42359.

Flagged Revisions is an important aspect of wikis and we know very little on how well it goes.
Some wikis keep a short validation queue, some an extremely long one.
Regular reports on this, maybe just showing the same data as Special:ValidationStatistics (which by the way is broken on some wikis right now) over time and in cross.wiki comparisons, would be extremely useful to understand the communiies' health.


Version: unspecified
Severity: enhancement

Details

Reference
bz42360
bzimport raised the priority of this task from to Lowest.
bzimport set Reference to bz42360.
bzimport added a subscriber: Unknown Object (MLST).
Nemo_bis created this task.Nov 22 2012, 2:15 PM

According to all available information, Wikimetrics deals with cohorts; this bug has nothing to do with that.

Elitre added a subscriber: Elitre.Feb 3 2015, 3:23 PM
Zache added a subscriber: Zache.Apr 15 2017, 6:26 PM
Restricted Application added a project: Analytics. · View Herald TranscriptApr 15 2017, 6:26 PM
Nuria added a subscriber: Nuria.Apr 17 2017, 3:26 PM

Ping @Nemo_bis is this still relevant?

Nuria moved this task from Incoming to Radar on the Analytics board.Apr 17 2017, 3:26 PM
Zache added a comment.Apr 18 2017, 2:48 AM

Still relevant. Current Special:ValidationStatistics is also broken; see bug T163107 .

Zache added a comment.Apr 30 2017, 9:05 AM

In example for tracking the effects of phab:T164049 some crosswiki stats would be nice.

Seems that this is related to our conversations about "measuring community backlog" cc @Milimetric

Nemo_bis added a comment.EditedMay 3 2017, 9:24 AM

Such a tool should only query the FlaggedRevs wikis' API for available statistics (also to be consistent with what the users see on their wikis). If an API doesn't exist for Special:ValidationStatistics, the first step would be to make one. (But we could also temporarily replicate the same queries on labsdb.)

I'm still a little lost here, I don't really know how FlaggedRevs or Special:ValidationStatistics work and what kind of data you need. Some more in-depth explanation would be useful.

Zache added a comment.EditedMay 4 2017, 4:36 PM

In fiwiki we are tracking couple key numbers.

  • maximum pending lag.
  • number of pending changes

Max pending lag doesn't really matter, but number of pending changes is important because if starts to grow it means that there is more edits that we can handle and something needs to be done. Second thing is that it would be important to get data to see if we change things to see how it will effect to reviewing. Only reason why average pendingLag is interesting in fiwiki is because it is easily available, but generally speaking max lag is more interesting.

Other numbers which in fiwiki context we check time to time is that how many active reviewers there is, how many manual reviews they are doing, how many reviews there is per reviewer in some time frame,. What percentage is the automatically/manually approved changes. This is because if number of automatically approved changes is too low or if the number of reviewers is too low then there is too much work to remaining reviewers and they will give up too and number of pending changes will start to rise.

When comparing to other wikis it would be nice to have better data how they are handling the review backlog. In example when we are comparing four diffirent wikipedias which are using FlaggedRevs pretty much in similar way there is clear differences in things like pendingLag-average (fiwiki, dewiki are fairly stable, huwiki, plwiki have zigzag-line)

And if we add ruwiki to that we can see that they just dont care about the pendingLag

But ruwiki is still second active in terms of number of reviews per day

(Note: The links above are for demonstration only and even if they are most likely correct the code which created the stats was old and broken so dont trust the graphs.)

Ok, thank you very much, I understand now, and I have added this topic of discussion in our feedback for the design / implementation of the new Wikistats 2.0 site: https://www.mediawiki.org/wiki/Topic:Tq5qpc3emof83azo

Initially we won't have time to collect, organize, and expose this data for the first version of the Wikistats rewrite. But we do have a major annual goal next fiscal year to examine community backlogs, and I think this work will fit nicely in there. Thanks for bringing it to our attention.