Page MenuHomePhabricator

Run refreshCategoryCounts.php on all WMF wikis
Closed, ResolvedPublic

Description

Author: happy.melon.wiki

Description:
Adds a new maintenance script refreshCategoryCounts.php

Following bug17155, the category population counts on WMF wikis aren't getting any *more* inaccurate, but they also aren't getting any more correct. A maintenance script needs to be run to refresh the cat_pages, cat_subcats and cat_files fields of the category table. I *think* that /maintenance/populateCategory.php would work, but that's not really what it was designed for, and it's certainly not the most efficient way of doing it. I've adapted it into a new script. So the "patch need-review" keywords are for someone to check that I haven't made some silly mistake on the patch, and commit it to svn. Then the "shell" is for someone to run it on all WMF wikis. If Simetrical did his job properly (:D) it should be easy on the servers.


Version: unspecified
Severity: enhancement

Attached:

Details

Reference
bz18488

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 21 2014, 10:31 PM
bzimport set Reference to bz18488.
bzimport added a subscriber: Unknown Object (MLST).

Maintenance script committed in r66140

I've run this script on all wikis except Commons over the past 2 days. Attempting to run it on Commons during peak time results in a database deadlock error before it even gets to 1000 categories, presumably because there's so much category-related activity going on on Commons. I will see about running it at a quieter time.

Made a modification suggested by Aryeh Gregor and ran the script on Commons this morning (UTC). Completed with only a single deadlock error.

Ankry added a subscriber: Ankry.Dec 11 2015, 8:23 PM
Restricted Application added a subscriber: Matanya. · View Herald TranscriptDec 11 2015, 8:23 PM
According the #wikimedia-operations log from 2012-02-03:

[14:07:19] <Reedy> mutante, said maintenance script is on bug 18488
[14:07:34] <Reedy> added in r66140 and reverted
[14:07:39] <mutante> ouch, i hope not in a way that using it breaks stuff more :)
[14:07:44] <Reedy> Try populateCategory.php

TTO added a subscriber: TTO.EditedFeb 1 2017, 3:02 PM

I wrote https://gerrit.wikimedia.org/r/333917... is that of any use? populateCategory.php is going to try to repopulate everything, even those whose counts are correct.

Edit: See T18765: Write a maintenance script to refresh category member counts and related tasks

This task is eight years old and it has been closed for seven of them. I think y'all want to be posting this on T18765.