Flagged revs Special:ValidationStatistics update doesn't work
Open, NormalPublic

Description

Flagged revision page review statistics (Special:ValidationStatistics) timestamp "The following data was last updated on 17 April 2017 at 08:07. " tells when page is generated. However the data which is shown on the page can be old .

In example Hungarian Wikipedias page reviews statistics says that "The average wait for edits by users that have not logged in to be reviewed is 25 h 59 min; the median is 3 h 38 min. " and based on db average 25.59 has been same from 14.4.2015 and median 3 h 38 from 13.12.2014. The data in percentile table is also years old.

Based on tool labs database case is that in backend stats are updated but for some reason something keeps inserting identical values to database. Bug effects least all Wikipedias with Flagged revs enabled.

Zache created this task.Apr 17 2017, 12:12 PM
Restricted Application added subscribers: TerraCodes, Aklapper. · View Herald TranscriptApr 17 2017, 12:12 PM

Some timestamps from the DB

reviewLag-anon-median
Quarry: https://quarry.wmflabs.org/query/18063

+-----------+-----------------------+----------------+----------------------------------+--------------+
| wiki      | frs_stat_key          | frs_timestamp  | oldest_timestamp_with_this_value | frs_stat_val |
+-----------+-----------------------+----------------+----------------------------------+--------------+
| alswiki_p | reviewLag-anon-median | 20170417120002 | 20121115230421                   |         1274 |
| ruwiki_p  | reviewLag-anon-median | 20170417100730 | 20161202182402                   |      1174770 |
| dewiki_p  | reviewLag-anon-median | 20170417120040 | 20161021181719                   |         7309 |
| plwiki_p  | reviewLag-anon-median | 20170417100711 | 20140523221756                   |         5689 |
| eowiki_p  | reviewLag-anon-median | 20170417100422 | 20121116063111                   |        13615 |
| huwiki_p  | reviewLag-anon-median | 20170417100658 | 20141213061507                   |        13113 |
| arwiki_p  | reviewLag-anon-median | 20170417120005 | 20131116140010                   |        96130 |
| iawiki_p  | reviewLag-anon-median | 20170417100659 | 20121116074414                   |      1206663 |
| hiwiki_p  | reviewLag-anon-median | 20170417100655 | 20121116072430                   |            0 |
| sqwiki_p  | reviewLag-anon-median | 20170417100738 | 20121116113209                   |        26144 |
| idwiki_p  | reviewLag-anon-median | 20170417100701 | 20140824141611                   |       725916 |
| kawiki_p  | reviewLag-anon-median | 20170417100703 | 20170325140649                   |         4713 |
| fiwiki_p  | reviewLag-anon-median | 20170417100652 | 20130223200302                   |         1115 |
| bnwiki_p  | reviewLag-anon-median | 20170417120008 | 20121116000223                   |            0 |
| trwiki_p  | reviewLag-anon-median | 20170417100743 | 20170408220809                   |        25667 |
| ukwiki_p  | reviewLag-anon-median | 20170417100750 | 20170415180832                   |      1289207 |
| vecwiki_p | reviewLag-anon-median | 20170417100752 | 20160923040721                   |        40167 |
| bewiki_p  | reviewLag-anon-median | 20170417120007 | 20170205140012                   |        54466 |
| enwiki_p  | reviewLag-anon-median | 20170417100059 | 20121116055234                   |            0 |
| ptwiki_p  | reviewLag-anon-median | 20170417100718 | 20131011201949                   |            0 |
| fawiki_p  | reviewLag-anon-median | 20170417100556 | 20140522201549                   |            0 |
| cewiki_p  | reviewLag-anon-median | 20170417120011 | 20160228180014                   |        32295 |
| mkwiki_p  | reviewLag-anon-median | 20170417100705 | 20130513160157                   |         6873 |
+-----------+-----------------------+----------------+----------------------------------+--------------+

reviewLag-anon-average
https://quarry.wmflabs.org/query/18064

+-----------+------------------------+----------------+----------------------------------+--------------+
| wiki      | frs_stat_key           | frs_timestamp  | oldest_timestamp_with_this_value | frs_stat_val |
+-----------+------------------------+----------------+----------------------------------+--------------+
| alswiki_p | reviewLag-anon-average | 20170417120002 | 20121115230421                   |       118624 |
| ruwiki_p  | reviewLag-anon-average | 20170417100730 | 20161202182402                   |      1719487 |
| dewiki_p  | reviewLag-anon-average | 20170417120040 | 20161021181719                   |        28481 |
| plwiki_p  | reviewLag-anon-average | 20170417100711 | 20160714160525                   |        34551 |
| eowiki_p  | reviewLag-anon-average | 20170417100422 | 20121116063111                   |       417771 |
| huwiki_p  | reviewLag-anon-average | 20170417100658 | 20150414181716                   |        93577 |
| arwiki_p  | reviewLag-anon-average | 20170417120005 | 20161012080008                   |       304493 |
| iawiki_p  | reviewLag-anon-average | 20170417100659 | 20121116074414                   |       892960 |
| hiwiki_p  | reviewLag-anon-average | 20170417100655 | 20121116072430                   |            0 |
| sqwiki_p  | reviewLag-anon-average | 20170417100738 | 20131201142208                   |        17419 |
| idwiki_p  | reviewLag-anon-average | 20170417100701 | 20140824141611                   |      3468755 |
| kawiki_p  | reviewLag-anon-average | 20170417100703 | 20170416180716                   |       133599 |
| fiwiki_p  | reviewLag-anon-average | 20170417100652 | 20130223200302                   |         6196 |
| bnwiki_p  | reviewLag-anon-average | 20170417120008 | 20121116000223                   |            0 |
| trwiki_p  | reviewLag-anon-average | 20170417100743 | 20170417100743                   |       392874 |
| ukwiki_p  | reviewLag-anon-average | 20170417100750 | 20170415180832                   |     10547028 |
| vecwiki_p | reviewLag-anon-average | 20170417100752 | 20160923040721                   |       192806 |
| bewiki_p  | reviewLag-anon-average | 20170417120007 | 20170207120010                   |       279930 |
| enwiki_p  | reviewLag-anon-average | 20170417120057 | 20121116055234                   |            0 |
| ptwiki_p  | reviewLag-anon-average | 20170417100718 | 20131011201949                   |            0 |
| fawiki_p  | reviewLag-anon-average | 20170417100556 | 20140522201549                   |            0 |
| cewiki_p  | reviewLag-anon-average | 20170417120011 | 20160228180014                   |      5751210 |
| mkwiki_p  | reviewLag-anon-average | 20170417100705 | 20130513160157                   |       496827 |
+-----------+------------------------+----------------+----------------------------------+--------------+

Problem is most likely related to backend/FlaggedRevsStats.php: getEditReviewTimes() -function.

Zache renamed this task from Flagged revs page review statistics update doesn't work to Flagged revs Special:ValidationStatistics update doesn't work.Apr 18 2017, 2:28 AM
Zache updated the task description. (Show Details)

Are those queries you made in quarry the same that FlaggedRevs uses?

The table with the breakdown by namespaces and the table with usernames are correct, aren't they? I can't see on hu.wiki because I don't have enough permissions, but on https://fi.wikipedia.org/wiki/Special:ValidationStatistics the numbers match with what I see in the linked special pages (e.g. 13 unreviewed templates).

Zache added a comment.May 3 2017, 10:07 AM

Database table is same that FlaggedRevs uses but queries are my own and i didn't check the FR code when i made them.

The breakdown by namespaces and the table with usernames should be correct. Also "The average review delay for pages with edits currently pending review" ( pendingLag-average in database) should be ok too.

All other review delay breakdowns are broken. Also Number of users with Editor/autoreviewer rights is broken in some wiki configs but when it works the number is correct. (T163102)

Change 354608 had a related patch set uploaded (by Nemo bis; owner: Federico Leva):
[operations/mediawiki-config@master] Restore default $wgFlaggedRevsStatsAge (2 hours)

https://gerrit.wikimedia.org/r/354608

This is the query FlaggedRevsStats::getMeanPendingEditTime() and it gives the same results as shown on https://ia.wikipedia.org/wiki/Special:Statisticas_de_validation for instance, as you said.

SELECT AVG( NOW() - UNIX_TIMESTAMP(fp_pending_since) )/3600/24 as avgDaysWait
FROM flaggedpages
JOIN page
ON fp_pending_since IS NOT NULL
AND fp_page_id = page_id
AND page_namespace IN (0, 6, 10);

Change 354614 had a related patch set uploaded (by Nemo bis; owner: Federico Leva):
[mediawiki/extensions/FlaggedRevs@master] Add some debugging output to FlaggedRevsStats

https://gerrit.wikimedia.org/r/354614

Nemo_bis triaged this task as Normal priority.May 20 2017, 12:58 AM

Change 354615 had a related patch set uploaded (by Nemo bis; owner: Federico Leva):
[operations/mediawiki-config@master] Enable ValidationStatistics log for FlaggedRevs

https://gerrit.wikimedia.org/r/354615