Page MenuHomePhabricator

Write a maintenance script to refresh category member counts
Closed, ResolvedPublic

Description

Currently there's no mechanism for refreshing category member counts. So certain categories, like http://en.wikipedia.org/wiki/Category:Temporary_Wikipedian_userpages say they have over 12,900 members, when in reality it has only about 3,000 members.

populateCategory.php is a bit of a brute-force approach (it recalculates everything, even categories whose counts are correct). We could do with another script that specifically recalculates wrong counts in the category table.

T85696 is about adding a way for users to invoke recounting when required.

Details

Reference
bz16765

Event Timeline

bzimport raised the priority of this task from to Low.Nov 21 2014, 10:28 PM
bzimport added a project: MediaWiki-General.
bzimport set Reference to bz16765.
bzimport added a subscriber: Unknown Object (MLST).

matthew.britton wrote:

Why admin-only?

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 13 2016, 10:15 AM
TTO claimed this task.Jan 24 2017, 11:53 AM
TTO added a subscriber: TTO.

I'm working on a maintenance script to make this happen. T85696: Allow action=purge to recalculate the number of pages/subcats/files in a category would be nice, and finding out the reason for the drift would be even nicer, but a server-side script is better than nothing.

TTO updated the task description. (Show Details)Jan 24 2017, 11:54 AM
TTO removed a subscriber: wikibugs-l-list.
TTO renamed this task from Need a method to refresh category member counts to Write a maintenance script to refresh category member counts.Jan 24 2017, 12:22 PM
TTO updated the task description. (Show Details)

Change 333917 had a related patch set uploaded (by TTO):
New maintenance script to recalculate category membership counts

https://gerrit.wikimedia.org/r/333917

You didn't like the "Basing it on a percentage of pageviews (.01% of page views purge the count)" suggestion? The idea was that you could re-calculate the category membership counts when random.randint(1, 10000) % 5 == 0 or str(random.random()).endswith('5') or something like that. Obviously most requests don't hit MediaWiki, but enough do that this would probably work for at least popular categories.

Change 333917 merged by jenkins-bot:
[mediawiki/core@master] New maintenance script to recalculate category membership counts

https://gerrit.wikimedia.org/r/333917

TTO closed this task as Resolved.Jul 15 2017, 9:24 AM
TTO added a subscriber: aaron.

@MZMcBride: Oh, I see what you meant. It would be interesting to know what @aaron thinks of that idea. In any case, if you'd like to pursue it, you could file a separate task.

TTO changed the status of subtask T170737: Run recountCategories.php on Wikimedia wikis from Open to Stalled.Jul 16 2017, 11:46 AM
TTO changed the status of subtask T170737: Run recountCategories.php on Wikimedia wikis from Stalled to Open.Jul 22 2017, 1:09 AM