Page MenuHomePhabricator

Run recountCategories.php on Wikimedia wikis
Open, LowPublic

Description

Run the new recountCategories.php script on all wikis. It has to be run three times on all wikis, once with --mode pages, once with --mode files and once with --mode subcats. You might want to pick a suitable value for --throttle as well.

Event Timeline

TTO created this task.Jul 15 2017, 9:36 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 15 2017, 9:36 AM
Reedy added a comment.Jul 15 2017, 2:38 PM

Can't be run until .10 is everywhere

TTO changed the task status from Open to Stalled.Jul 16 2017, 11:46 AM
TTO changed the task status from Stalled to Open.Jul 22 2017, 1:09 AM

.10 is now everywhere :)

Samtar added a subscriber: Samtar.Aug 25 2017, 11:49 AM
TTO added a comment.Jan 16 2018, 3:28 AM

@Reedy any chance you will have time to look at running this?

Reedy added a comment.Jan 16 2018, 4:02 PM

Fun, yeah, I can.

Will look at running it on the test wikis today, and see how the output looks etc :)

Reedy added a comment.Jan 16 2018, 4:31 PM
reedy@terbium:~$ mwscript recountCategories.php --wiki=testwiki --mode=pages | tee ~/testwiki.log
Finding up to 500 drifted rows starting at cat_id 500...
Updating cat_pages field on 154 rows...
Finding up to 500 drifted rows starting at cat_id 500...
Done! Updated the pages counts of 154 categories.
Now run the script using the other --mode options if you haven't already.
Also run 'php cleanupEmptyCategories.php --mode remove' to remove empty,
nonexistent categories from the category table.

reedy@terbium:~$ mwscript recountCategories.php --wiki=testwiki --mode=subcats | tee ~/testwiki.log
Finding up to 500 drifted rows starting at cat_id 500...
Updating cat_subcats field on 4 rows...
Finding up to 500 drifted rows starting at cat_id 500...
Done! Updated the subcats counts of 4 categories.
Now run the script using the other --mode options if you haven't already.
reedy@terbium:~$ mwscript recountCategories.php --wiki=testwiki --mode=files | tee ~/testwiki.log
Finding up to 500 drifted rows starting at cat_id 500...
Updating cat_files field on 10 rows...
Finding up to 500 drifted rows starting at cat_id 500...
Done! Updated the files counts of 10 categories.
Now run the script using the other --mode options if you haven't already.
reedy@terbium:~$ mwscript cleanupEmptyCategories.php --wiki=testwiki | tee ~/testwiki.log
...Update 'cleanup empty categories' already logged as completed.
reedy@terbium:~$ mwscript cleanupEmptyCategories.php --wiki=testwiki --force | tee ~/testwiki.log
Adding empty categories with description pages...
Removing empty categories without description pages...
The category named :Sub-Sub-Category_Bleah_tst is not valid?!
--mode=remove --begin=‪中文(简体)‬
Category cleanup complete.
reedy@terbium:~$
Reedy added a comment.Jan 16 2018, 4:32 PM
reedy@terbium:~$ mwscript recountCategories.php --wiki=test2wiki --mode=pages | tee ~/test2wiki.log
Finding up to 500 drifted rows starting at cat_id 500...
Updating cat_pages field on 25 rows...
Finding up to 500 drifted rows starting at cat_id 500...
Done! Updated the pages counts of 25 categories.
Now run the script using the other --mode options if you haven't already.
Also run 'php cleanupEmptyCategories.php --mode remove' to remove empty,
nonexistent categories from the category table.

reedy@terbium:~$ mwscript recountCategories.php --wiki=test2wiki --mode=subcats | tee ~/test2wiki.log
Finding up to 500 drifted rows starting at cat_id 500...
Done! Updated the subcats counts of 0 categories.
Now run the script using the other --mode options if you haven't already.
reedy@terbium:~$ mwscript recountCategories.php --wiki=test2wiki --mode=files | tee ~/test2wiki.log
Finding up to 500 drifted rows starting at cat_id 500...
Done! Updated the files counts of 0 categories.
Now run the script using the other --mode options if you haven't already.
reedy@terbium:~$ mwscript cleanupEmptyCategories.php --wiki=test2wiki --force | tee ~/test2wiki.log
Adding empty categories with description pages...
Removing empty categories without description pages...
--mode=remove --begin=Statut_UICN_EN
Category cleanup complete.
reedy@terbium:~$
TTO added a subscriber: MaxSem.Jan 17 2018, 7:57 PM

Unfortunately line 101 of the script is wrong. It should be printing $this->minimumId. I don't know if that really matters though, it looks like that was just intended to let you see how long the script is taking, and I guess you'll be running it headless.

Ping @MaxSem regardless.