Page MenuHomePhabricator
Create Task
Maniphest T440023

Category file counts display negative page counts
Closed, ResolvedPublicBUG REPORT
Actions

Description

Steps to replicate the issue (include links if applicable):

  • Go to any category with subcategory and look at the children count

What happens?:
The values seems compeltely erratic since yesterday. It seems some files are counted as negative page counts. Leading to completely invalid counts, with categories without page displaying a negative number of pages

Example here.

 Milimedia files, Belgium (review needed) (-44 P, 5714 F)
...
 Milimedia files, Japan (review needed) (-3445 P, 53265 F)
...
 Milimedia files, Slovakia (review needed) (144 P)  <<< this category is empty, yet displaying 144 pages out of nowhere ?!

image.png (2,314×2,533 px, 517 KB)

I see this behaviour almost everywhere.

What should have happened instead?:

The file count should be exact, as before. Having wrong counts for maintenance categories is really harmful.

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
recountCategories: Use virtual domain for categorylinks queryZabemediawiki/coremaster+80 -40
Customize query in gerrit

Related Objects

Event Timeline

Don-vip created this task.Fri, Oct 2, 5:11 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFri, Oct 2, 5:11 PM
Don-vip updated the task description. (Show Details)Fri, Oct 2, 5:30 PM
Don-vip updated the task description. (Show Details)
Peachey88 subscribed.Fri, Oct 2, 10:03 PM

This could potentially be T439902: Wikimedia\Rdbms\DBQueryError: Error 1205: Lock wait timeout exceeded; try restarting transactionFunction: MediaWiki\JobQueue\Jobs\CategoryCountUpdateJob::updateCategoryCountsQuery: UPDATE `category` SET cat_pages = cat_pages

Kallichore subscribed.Fri, Oct 2, 10:09 PM

Using https://commons.wikimedia.org/w/api.php?action=query&prop=categoryinfo&titles=Category:Milimedia%20files,%20Panama%20(review%20needed)&format=json I got for some time:

{"batchcomplete":"","query":{"pages":{"169502072":{"pageid":169502072,"ns":14,"title":"Category:Milimedia files, Panama (review needed)","categoryinfo":{"size":5356,"pages":-314,"files":5670,"subcats":0}}}}}

However right now "size" is 5670 again and everything is consistent.

Umherirrender added subscribers: Zabe, Ladsgroup, Umherirrender.Sat, Oct 3, 8:57 AM

At the begin of each month the maintenance script recountCategories.php is running, which could change this numbers.
Maybe the script run on commons does not find the new links tables on x4 (T398709: FY2025-26 WE 6.4.1: Move links tables of commons to a dedicated cluster) and counts old data.

Kallichore added a comment.Sat, Oct 3, 9:18 AM

I found that the problem is fixed by just opening a problematic category. The parent category shows consistent numbers afer action=purge then.

Sat, 03 Oct 2026 09:07:54 GMT (before I opened Category:Milimedia files, Indian Ocean (review needed) ):

{"batchcomplete":"","query":{"normalized":[{"from":"Category:Milimedia files, Indian Ocean (review needed) ","to":"Category:Milimedia files, Indian Ocean (review needed)"}],"pages":{"163210719":{"pageid":163210719,"ns":14,"title":"Category:Milimedia files, Indian Ocean (review needed)","categoryinfo":{"size":356,"pages":-162,"files":518,"subcats":0}}}}}

Sat, 03 Oct 2026 09:08:49 GMT (after I opened Category:Milimedia files, Indian Ocean (review needed) ):

{"batchcomplete":"","query":{"normalized":[{"from":"Category:Milimedia files, Indian Ocean (review needed) ","to":"Category:Milimedia files, Indian Ocean (review needed)"}],"pages":{"163210719":{"pageid":163210719,"ns":14,"title":"Category:Milimedia files, Indian Ocean (review needed)","categoryinfo":{"size":518,"pages":0,"files":518,"subcats":0}}}}}
Umherirrender added a comment.Sat, Oct 3, 9:45 AM

action=purge refresh the counts if there are not more as 5000 members in the category.
Bigger categories get not "fixed" with purge (for performance reasons). The script needs to be rerun after fixing (or waiting for regular run on next month).

gerritbot added a comment.Sat, Oct 3, 4:00 PM

Change #1349998 had a related patch set uploaded (by Zabe; author: Zabe):

[mediawiki/core@master] recountCategories: Use virtual domain for categorylinks query

https://gerrit.wikimedia.org/r/1349998

gerritbot added a project: Patch-For-Review.Sat, Oct 3, 4:00 PM
gerritbot added a comment.Sun, Oct 4, 12:32 AM

Change #1349998 merged by jenkins-bot:

[mediawiki/core@master] recountCategories: Use virtual domain for categorylinks query

https://gerrit.wikimedia.org/r/1349998

ReleaseTaggerBot added a project: MW-1.47-notes (1.47.0-wmf.23; 2026-10-06).Sun, Oct 4, 1:00 AM
Maintenance_bot removed a project: Patch-For-Review.Sun, Oct 4, 1:30 AM
Aklapper added a project: MediaWiki-Maintenance-system.Tue, Oct 6, 8:41 AM
Stashbot added a comment.Thu, Oct 8, 12:49 PM

Mentioned in SAL (#wikimedia-operations) [2026-10-08T12:49:12Z] <zabe> zabe@deploy1003:~$ mwscript recountCategories.php commonswiki --mode all # T440023 T439902

Stashbot mentioned this in T439902: Wikimedia\Rdbms\DBQueryError: Error 1205: Lock wait timeout exceeded; try restarting transactionFunction: MediaWiki\JobQueue\Jobs\CategoryCountUpdateJob::updateCategoryCountsQuery: UPDATE `category` SET cat_pages = cat_pages.Thu, Oct 8, 12:49 PM
Zabe added a comment.EditedThu, Oct 8, 1:03 PM

recountCategories.php failed with T439902: Wikimedia\Rdbms\DBQueryError: Error 1205: Lock wait timeout exceeded; try restarting transactionFunction: MediaWiki\JobQueue\Jobs\CategoryCountUpdateJob::updateCategoryCountsQuery: UPDATE `category` SET cat_pages = cat_pages.

Finding up to 500 drifted rows greater than cat_id 228482501...
Updating cat_pages field on up to 3 rows...
Finding up to 500 drifted rows greater than cat_id 228510562...
Updating cat_pages field on up to 4 rows...
Finding up to 500 drifted rows greater than cat_id 228538962...
Updating cat_pages field on up to 6 rows...
Finding up to 500 drifted rows greater than cat_id 228569464...
Updating cat_pages field on up to 5 rows...
Wikimedia\Rdbms\DBQueryError from line 1232 of /srv/mediawiki-staging/php-1.47.0-wmf.23/includes/libs/Rdbms/Database/Database.php: Error 1205: Lock wait timeout exceeded; try restarting transaction
Function: RecountCategories::doWork
Query: UPDATE  `category` SET cat_pages = 35447028 WHERE cat_id = 228605640 AND (cat_pages != 35447028)

#0 /srv/mediawiki-staging/php-1.47.0-wmf.23/includes/libs/Rdbms/Database/Database.php(1216): Wikimedia\Rdbms\Database->getQueryException('Lock wait timeo...', 1205, 'UPDATE  `catego...', 'RecountCategori...')
#1 /srv/mediawiki-staging/php-1.47.0-wmf.23/includes/libs/Rdbms/Database/Database.php(1190): Wikimedia\Rdbms\Database->getQueryExceptionAndLog('Lock wait timeo...', 1205, 'UPDATE  `catego...', 'RecountCategori...')
#2 /srv/mediawiki-staging/php-1.47.0-wmf.23/includes/libs/Rdbms/Database/Database.php(645): Wikimedia\Rdbms\Database->reportQueryError('Lock wait timeo...', 1205, 'UPDATE  `catego...', 'RecountCategori...', false)
#3 /srv/mediawiki-staging/php-1.47.0-wmf.23/includes/libs/Rdbms/Database/Database.php(1533): Wikimedia\Rdbms\Database->query(Object(Wikimedia\Rdbms\Query), 'RecountCategori...')
#4 /srv/mediawiki-staging/php-1.47.0-wmf.23/includes/libs/Rdbms/Database/DBConnRef.php(129): Wikimedia\Rdbms\Database->update('category', Array, Array, 'RecountCategori...', Array)
#5 /srv/mediawiki-staging/php-1.47.0-wmf.23/includes/libs/Rdbms/Database/DBConnRef.php(459): Wikimedia\Rdbms\DBConnRef->__call('update', Array)
#6 /srv/mediawiki-staging/php-1.47.0-wmf.23/includes/libs/Rdbms/QueryBuilder/UpdateQueryBuilder.php(311): Wikimedia\Rdbms\DBConnRef->update('category', Array, Array, 'RecountCategori...', Array)
#7 /srv/mediawiki-staging/php-1.47.0-wmf.23/maintenance/recountCategories.php(229): Wikimedia\Rdbms\UpdateQueryBuilder->execute()
#8 /srv/mediawiki-staging/php-1.47.0-wmf.23/maintenance/recountCategories.php(88): RecountCategories->doWork('pages')
#9 /srv/mediawiki-staging/php-1.47.0-wmf.23/maintenance/includes/MaintenanceRunner.php(695): RecountCategories->execute()
#10 /srv/mediawiki-staging/php-1.47.0-wmf.23/maintenance/run.php(54): MediaWiki\Maintenance\MaintenanceRunner->run()
#11 /srv/mediawiki-staging/multiversion/MWScript.php(219): require_once('/srv/mediawiki-...')
#12 {main}
zabe@deploy1003:~$

Btw 228605640 is the cat_id of the category CC-BY-SA-4.0, so 35447028 is actually the correct number of pages in it.

Zabe closed this task as Resolved.Sat, Oct 10, 4:47 PM
Zabe claimed this task.

A second run of recountCategories.php went through.

[...]
Updated the files counts of 11914 categories.
[...]

Please reopen this task if you can still see wrong counts somewhere.

Zabe added a project: DBA.Sat, Oct 10, 4:47 PM
Zabe moved this task from Triage to Done on the DBA board.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits