Page MenuHomePhabricator

Convert various core/extension cache users to the WAN cache as needed
Closed, ResolvedPublic

Description

Per http://www.mediawiki.org/wiki/Requests_for_comment/Master_%26_slave_datacenter_strategy_for_MediaWiki#Design_implications

This could even be part of some hackathon type sprinting :)

Patches could not be merged until the new cache was up and configure of course.

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Gilles added a subscriber: Gilles.Apr 2 2015, 12:24 PM

Change 206975 had a related patch set uploaded (by Aaron Schulz):
Converted sidebar cache to the WAN cache

https://gerrit.wikimedia.org/r/206975

Change 206968 had a related patch set uploaded (by Aaron Schulz):
Converted LocalRepo to using the WAN cache

https://gerrit.wikimedia.org/r/206968

Change 206989 had a related patch set uploaded (by Aaron Schulz):
Converted InfoAction to using the WAN cache

https://gerrit.wikimedia.org/r/206989

Change 206998 had a related patch set uploaded (by Aaron Schulz):
Converted User touched key to using the WAN cache

https://gerrit.wikimedia.org/r/206998

Change 207001 had a related patch set uploaded (by Aaron Schulz):
Converted User object cache to the WAN cache

https://gerrit.wikimedia.org/r/207001

Change 207004 had a related patch set uploaded (by Aaron Schulz):
Converted LocalFile to using the WAN cache

https://gerrit.wikimedia.org/r/207004

Change 206966 had a related patch set uploaded (by Aaron Schulz):
Converted ChangeTags to using the WAN cache

https://gerrit.wikimedia.org/r/206966

Change 206958 had a related patch set uploaded (by Aaron Schulz):
Converted FileBackendStore to using the WAN cache

https://gerrit.wikimedia.org/r/206958

Change 207638 had a related patch set uploaded (by Aaron Schulz):
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/207638

Change 207639 had a related patch set uploaded (by Aaron Schulz):
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/207639

Change 207646 had a related patch set uploaded (by Aaron Schulz):
Converted Interwiki using WAN cache

https://gerrit.wikimedia.org/r/207646

Change 207652 had a related patch set uploaded (by Aaron Schulz):
Converted WikiSet to using the WAN cache

https://gerrit.wikimedia.org/r/207652

Change 207630 had a related patch set uploaded (by Aaron Schulz):
Use the WAN cache for caching

https://gerrit.wikimedia.org/r/207630

aaron moved this task from Backlog to Doing on the Availability board.Apr 30 2015, 5:24 AM

Change 207729 had a related patch set uploaded (by Aaron Schulz):
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/207729

Change 207730 had a related patch set uploaded (by Aaron Schulz):
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/207730

Change 207891 had a related patch set uploaded (by Aaron Schulz):
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/207891

Change 207897 had a related patch set uploaded (by Aaron Schulz):
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/207897

Change 207915 had a related patch set uploaded (by Aaron Schulz):
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/207915

Change 207638 merged by jenkins-bot:
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/207638

Change 207639 merged by jenkins-bot:
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/207639

Change 207897 merged by jenkins-bot:
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/207897

Change 206989 merged by jenkins-bot:
Converted InfoAction to using the WAN cache

https://gerrit.wikimedia.org/r/206989

Change 207915 merged by jenkins-bot:
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/207915

Change 207891 merged by jenkins-bot:
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/207891

Change 206966 merged by jenkins-bot:
Converted ChangeTags to using the WAN cache

https://gerrit.wikimedia.org/r/206966

Change 207652 merged by jenkins-bot:
Converted WikiSet to using the WAN cache

https://gerrit.wikimedia.org/r/207652

Change 206968 merged by jenkins-bot:
Converted LocalRepo to using the WAN cache

https://gerrit.wikimedia.org/r/206968

Change 207630 merged by jenkins-bot:
Use the WAN cache for caching

https://gerrit.wikimedia.org/r/207630

Change 206975 merged by jenkins-bot:
Converted sidebar cache to the WAN cache

https://gerrit.wikimedia.org/r/206975

Change 207646 merged by jenkins-bot:
Converted Interwiki using WAN cache

https://gerrit.wikimedia.org/r/207646

Please see comment in commit 206966.

Change 207004 merged by jenkins-bot:
Converted LocalFile to using the WAN cache

https://gerrit.wikimedia.org/r/207004

Change 206998 merged by jenkins-bot:
Converted User touched key to using the WAN cache

https://gerrit.wikimedia.org/r/206998

Change 207729 merged by jenkins-bot:
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/207729

Change 210647 had a related patch set uploaded (by Aaron Schulz):
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/210647

Change 210647 merged by jenkins-bot:
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/210647

aaron moved this task from Doing to Blocked on the Availability board.May 15 2015, 2:58 AM

Change 207001 merged by jenkins-bot:
Converted User object cache to the WAN cache

https://gerrit.wikimedia.org/r/207001

Change 206958 merged by jenkins-bot:
Converted FileBackendStore to using the WAN cache

https://gerrit.wikimedia.org/r/206958

Change 207730 merged by jenkins-bot:
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/207730

Change 213186 had a related patch set uploaded (by Ori.livneh):
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/213186

Change 213186 merged by jenkins-bot:
Conversion to using WAN cache

https://gerrit.wikimedia.org/r/213186

Please see comment in commit 206966.

Since the delete operation of WANObjectCache prevents any caching for ten seconds, this had the effect of preventing any caching of tag usage statistics whenever a tag is added (or removed) for the next ten seconds.
On highly active wikis like wikipedias, where tags are added every few seconds, this means that caching would almost never occur. In all likelihood, this would make Special:Tags unusable.

https://en.wikipedia.org/wiki/Special:Tags WFM, is all well?

aaron added a comment.May 24 2015, 7:44 AM

Please see comment in commit 206966.
Since the delete operation of WANObjectCache prevents any caching for ten seconds, this had the effect of preventing any caching of tag usage statistics whenever a tag is added (or removed) for the next ten seconds.
On highly active wikis like wikipedias, where tags are added every few seconds, this means that caching would almost never occur. In all likelihood, this would make Special:Tags unusable.

https://en.wikipedia.org/wiki/Special:Tags WFM, is all well?

A follow-up patch was made in 3782a0d53bbe to improve performance post-purges to avoid stampedes.

The query is still awful and the constant purges seem kind of stupid (it probably can just purge when the new count is 0 or 1). Also I wonder if it really needs to count 2mil items or if it can clip the count. Also self::tagUsageStatistics() could use a process cache since permission checks call it several times for each tag. Lots of room for improvement.

The query is still awful and the constant purges seem kind of stupid (it probably can just purge when the new count is 0 or 1). Also I wonder if it really needs to count 2mil items or if it can clip the count. Also self::tagUsageStatistics() could use a process cache since permission checks call it several times for each tag. Lots of room for improvement.

I've been trying to address these and other issues in commit 201905.

aaron closed this task as Resolved.Jul 2 2015, 7:27 PM

Change 225013 had a related patch set uploaded (by Aaron Schulz):
Conversion to WANObjectCache

https://gerrit.wikimedia.org/r/225013

Change 225013 merged by jenkins-bot:
Conversion to WANObjectCache

https://gerrit.wikimedia.org/r/225013