Page MenuHomePhabricator

Flagged revs Special:ValidationStatistics update doesn't work
Open, MediumPublic

Description

Flagged revision page review statistics (Special:ValidationStatistics) timestamp "The following data was last updated on 17 April 2017 at 08:07. " tells when page is generated. However the data which is shown on the page can be old .

In example Hungarian Wikipedias page reviews statistics says that "The average wait for edits by users that have not logged in to be reviewed is 25 h 59 min; the median is 3 h 38 min. " and based on db average 25.59 has been same from 14.4.2015 and median 3 h 38 from 13.12.2014. The data in percentile table is also years old.

Based on tool labs database case is that in backend stats are updated but for some reason something keeps inserting identical values to database. Bug effects least all Wikipedias with Flagged revs enabled.

Broken function is likely backend/FlaggedRevsStats.php:getEditReviewTimes() as only reviewLag-anon-* and reviewLag-user-* values in flaggedrevs-statistics table in database are broken. Other values are correclty logged to database.

Details

Related Gerrit Patches:
operations/mediawiki-config : masterRestore default $wgFlaggedRevsStatsAge (2 hours)
mediawiki/extensions/FlaggedRevs : masterCleanup cache update logic in SpecialValidationStatistics
mediawiki/extensions/FlaggedRevs : masterAdd some debugging output to FlaggedRevsStats
operations/mediawiki-config : masterEnable ValidationStatistics log for FlaggedRevs

Event Timeline

Zache created this task.Apr 17 2017, 12:12 PM
Restricted Application added subscribers: TerraCodes, Aklapper. · View Herald TranscriptApr 17 2017, 12:12 PM

Some timestamps from the DB

reviewLag-anon-median
Quarry: https://quarry.wmflabs.org/query/18063

+-----------+-----------------------+----------------+----------------------------------+--------------+
| wiki      | frs_stat_key          | frs_timestamp  | oldest_timestamp_with_this_value | frs_stat_val |
+-----------+-----------------------+----------------+----------------------------------+--------------+
| alswiki_p | reviewLag-anon-median | 20170417120002 | 20121115230421                   |         1274 |
| ruwiki_p  | reviewLag-anon-median | 20170417100730 | 20161202182402                   |      1174770 |
| dewiki_p  | reviewLag-anon-median | 20170417120040 | 20161021181719                   |         7309 |
| plwiki_p  | reviewLag-anon-median | 20170417100711 | 20140523221756                   |         5689 |
| eowiki_p  | reviewLag-anon-median | 20170417100422 | 20121116063111                   |        13615 |
| huwiki_p  | reviewLag-anon-median | 20170417100658 | 20141213061507                   |        13113 |
| arwiki_p  | reviewLag-anon-median | 20170417120005 | 20131116140010                   |        96130 |
| iawiki_p  | reviewLag-anon-median | 20170417100659 | 20121116074414                   |      1206663 |
| hiwiki_p  | reviewLag-anon-median | 20170417100655 | 20121116072430                   |            0 |
| sqwiki_p  | reviewLag-anon-median | 20170417100738 | 20121116113209                   |        26144 |
| idwiki_p  | reviewLag-anon-median | 20170417100701 | 20140824141611                   |       725916 |
| kawiki_p  | reviewLag-anon-median | 20170417100703 | 20170325140649                   |         4713 |
| fiwiki_p  | reviewLag-anon-median | 20170417100652 | 20130223200302                   |         1115 |
| bnwiki_p  | reviewLag-anon-median | 20170417120008 | 20121116000223                   |            0 |
| trwiki_p  | reviewLag-anon-median | 20170417100743 | 20170408220809                   |        25667 |
| ukwiki_p  | reviewLag-anon-median | 20170417100750 | 20170415180832                   |      1289207 |
| vecwiki_p | reviewLag-anon-median | 20170417100752 | 20160923040721                   |        40167 |
| bewiki_p  | reviewLag-anon-median | 20170417120007 | 20170205140012                   |        54466 |
| enwiki_p  | reviewLag-anon-median | 20170417100059 | 20121116055234                   |            0 |
| ptwiki_p  | reviewLag-anon-median | 20170417100718 | 20131011201949                   |            0 |
| fawiki_p  | reviewLag-anon-median | 20170417100556 | 20140522201549                   |            0 |
| cewiki_p  | reviewLag-anon-median | 20170417120011 | 20160228180014                   |        32295 |
| mkwiki_p  | reviewLag-anon-median | 20170417100705 | 20130513160157                   |         6873 |
+-----------+-----------------------+----------------+----------------------------------+--------------+

reviewLag-anon-average
https://quarry.wmflabs.org/query/18064

+-----------+------------------------+----------------+----------------------------------+--------------+
| wiki      | frs_stat_key           | frs_timestamp  | oldest_timestamp_with_this_value | frs_stat_val |
+-----------+------------------------+----------------+----------------------------------+--------------+
| alswiki_p | reviewLag-anon-average | 20170417120002 | 20121115230421                   |       118624 |
| ruwiki_p  | reviewLag-anon-average | 20170417100730 | 20161202182402                   |      1719487 |
| dewiki_p  | reviewLag-anon-average | 20170417120040 | 20161021181719                   |        28481 |
| plwiki_p  | reviewLag-anon-average | 20170417100711 | 20160714160525                   |        34551 |
| eowiki_p  | reviewLag-anon-average | 20170417100422 | 20121116063111                   |       417771 |
| huwiki_p  | reviewLag-anon-average | 20170417100658 | 20150414181716                   |        93577 |
| arwiki_p  | reviewLag-anon-average | 20170417120005 | 20161012080008                   |       304493 |
| iawiki_p  | reviewLag-anon-average | 20170417100659 | 20121116074414                   |       892960 |
| hiwiki_p  | reviewLag-anon-average | 20170417100655 | 20121116072430                   |            0 |
| sqwiki_p  | reviewLag-anon-average | 20170417100738 | 20131201142208                   |        17419 |
| idwiki_p  | reviewLag-anon-average | 20170417100701 | 20140824141611                   |      3468755 |
| kawiki_p  | reviewLag-anon-average | 20170417100703 | 20170416180716                   |       133599 |
| fiwiki_p  | reviewLag-anon-average | 20170417100652 | 20130223200302                   |         6196 |
| bnwiki_p  | reviewLag-anon-average | 20170417120008 | 20121116000223                   |            0 |
| trwiki_p  | reviewLag-anon-average | 20170417100743 | 20170417100743                   |       392874 |
| ukwiki_p  | reviewLag-anon-average | 20170417100750 | 20170415180832                   |     10547028 |
| vecwiki_p | reviewLag-anon-average | 20170417100752 | 20160923040721                   |       192806 |
| bewiki_p  | reviewLag-anon-average | 20170417120007 | 20170207120010                   |       279930 |
| enwiki_p  | reviewLag-anon-average | 20170417120057 | 20121116055234                   |            0 |
| ptwiki_p  | reviewLag-anon-average | 20170417100718 | 20131011201949                   |            0 |
| fawiki_p  | reviewLag-anon-average | 20170417100556 | 20140522201549                   |            0 |
| cewiki_p  | reviewLag-anon-average | 20170417120011 | 20160228180014                   |      5751210 |
| mkwiki_p  | reviewLag-anon-average | 20170417100705 | 20130513160157                   |       496827 |
+-----------+------------------------+----------------+----------------------------------+--------------+

Problem is most likely related to backend/FlaggedRevsStats.php: getEditReviewTimes() -function.

Zache renamed this task from Flagged revs page review statistics update doesn't work to Flagged revs Special:ValidationStatistics update doesn't work.Apr 18 2017, 2:28 AM
Zache updated the task description. (Show Details)

Are those queries you made in quarry the same that FlaggedRevs uses?

The table with the breakdown by namespaces and the table with usernames are correct, aren't they? I can't see on hu.wiki because I don't have enough permissions, but on https://fi.wikipedia.org/wiki/Special:ValidationStatistics the numbers match with what I see in the linked special pages (e.g. 13 unreviewed templates).

Zache added a comment.May 3 2017, 10:07 AM

Database table is same that FlaggedRevs uses but queries are my own and i didn't check the FR code when i made them.

The breakdown by namespaces and the table with usernames should be correct. Also "The average review delay for pages with edits currently pending review" ( pendingLag-average in database) should be ok too.

All other review delay breakdowns are broken. Also Number of users with Editor/autoreviewer rights is broken in some wiki configs but when it works the number is correct. (T163102)

Change 354608 had a related patch set uploaded (by Nemo bis; owner: Federico Leva):
[operations/mediawiki-config@master] Restore default $wgFlaggedRevsStatsAge (2 hours)

https://gerrit.wikimedia.org/r/354608

This is the query FlaggedRevsStats::getMeanPendingEditTime() and it gives the same results as shown on https://ia.wikipedia.org/wiki/Special:Statisticas_de_validation for instance, as you said.

SELECT AVG( NOW() - UNIX_TIMESTAMP(fp_pending_since) )/3600/24 as avgDaysWait
FROM flaggedpages
JOIN page
ON fp_pending_since IS NOT NULL
AND fp_page_id = page_id
AND page_namespace IN (0, 6, 10);

Change 354614 had a related patch set uploaded (by Nemo bis; owner: Federico Leva):
[mediawiki/extensions/FlaggedRevs@master] Add some debugging output to FlaggedRevsStats

https://gerrit.wikimedia.org/r/354614

Nemo_bis triaged this task as Medium priority.May 20 2017, 12:58 AM

Change 354615 had a related patch set uploaded (by Nemo bis; owner: Federico Leva):
[operations/mediawiki-config@master] Enable ValidationStatistics log for FlaggedRevs

https://gerrit.wikimedia.org/r/354615

Tufor added a comment.Jan 9 2020, 11:06 PM

@Aklapper: so I've read all that discussion and apparently this extension has no assigned owner/maintainer; therefore this bug won't be fixed in the months/years to come, right?

@Tufor: For the start, someone would have to review the patches linked in this task... but finding folks will be hard currently, yes.

Change 354615 abandoned by Jforrester:
Enable ValidationStatistics log for FlaggedRevs

Reason:
Dependency still isn't merged after three years.

https://gerrit.wikimedia.org/r/354615

Change 354614 abandoned by Jforrester:
Add some debugging output to FlaggedRevsStats

Reason:
Abandonware patch, no fix in three years.

https://gerrit.wikimedia.org/r/354614

change 354608:

Is this still wanted?

Updating the statistics more often is definitely wanted. I've not repeated the assessment above on how stale the data is. There's certainly still a lot of outdated configuration settings in the FlaggedRevs file.

Change 571853 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/extensions/FlaggedRevs@master] Cleanup cache update logic in SpecialValidationStatistics

https://gerrit.wikimedia.org/r/571853

Change 354608 abandoned by Jforrester:
Restore default $wgFlaggedRevsStatsAge (2 hours)

Reason:
Per Aaron.

https://gerrit.wikimedia.org/r/354608

Zache updated the task description. (Show Details)Fri, Feb 14, 9:06 AM