Page MenuHomePhabricator

Reduce and clarify ObjectCache entry points
Open, LowPublic

Description

We have too many different cache objects, configuration variables, methods and factories for various types of cache interfaces.

Entry points

A few of the things we use right now:

  • wfGetCache( "hash" ), new HashBagOStuff();.
  • wfGetCache( CACHE_ANYTHING ), ObjectCache::newAnything().
  • ObjectCache::newAccelerator( fallback ).
  • wfGetMainCache(), ObjectCache::getMainClusterInstance().
  • ObjectCache::getMainWANInstance().
  • ObjectCache::getMainStashInstance().
  • ObjectCache::getInstance().
  • ObjectCache::getWANInstance().
  • more..
  • variations on the above due to back-compat aliases.

We should reduce this to a handful only that we use everywhere.

With the multi-datacenter work (T88445) we're also changing which cache developers should use by default. Namely one should now default to WANCache instead of the local-dc cache. It may make sense to also update the notion of "main cache" to start referring to WAN cache to clarify this convention shift.

I propose we standardise on the following four main entry points:

  • Server cache or Process cache (e.g. APC, fallback to none, or hash.)
  • Local cluster cache (e.g. Memcached).
  • WAN cache (e.g. Memcached, with relayed purges and other improvements).
  • Main stash (e.g. db-replicated).

With two extra entry points for special cases:

  • Identified cache group (e.g. ObjectCache::getGroup( "language-converter" )).
  • Identified cache backend (should be unused, but for back-compat, e.g. ObjectCache::getInstance( CACHE_DB );

Reduced interface:

class ObjectCache {
 static function getInstance( string $type ); // Key from $wgObjectCaches

 static function getGroup( string $groupName ); // Key from $wgObjectCacheGroups, defaults to main cache

 static function newAccelerator();

 static function getLocalClusterCache();

 static function getMainCache();

 static function getMainStash();
}

Action items:

  • Don't require custom accelerator fallback. Set a default one (empty or hash) and deprecate the fallback parameter.
  • Give CACHE_ACCEL sensible default so it doesn't fail on plain installs.
  • Reduce ObjectCache entry points to just WAN, LocalServer (APC), LocalCluster (Main) and stash.
  • Deprecate wfGetCache(), wfGetMainCache().

Configuration

Some configuration:

  • $wgObjectCaches array( .. )
  • $wgWANObjectCaches array( .. )
  • $wgMainCacheType
  • $wgMainWANCache
  • $wgMainStash
  • $wgMessageCacheType, $wgParserCacheType, $wgSessionCacheType, $wgLanguageConverterCacheType (maybe introduce some kind of cache grouping to allow overriding backends for individual cache groups, defaulting to main cache).

Action items:

  • Deprecate cache group config vars ($wgMessageCacheType, $wgParserCacheType etc.) in favour of e.g. $wgObjectCacheGroups.
  • Implement ObjectCache::getGroup( $name ) with default to wgMainCacheType.
  • Deprecate wfGetMessageCacheStorage(), wfGetParserCacheStorage()

Once we have $wgObjectCacheGroups, it'll be less complicated to introduce new cache groups. Making it easy to move around things in wmf-production and re-use caches without having to add more wg*CacheType variables and ad-hoc instantiation of BagOStuff objects.

See also:

Event Timeline

Krinkle created this task.Oct 19 2015, 5:14 PM
Krinkle raised the priority of this task from to Needs Triage.
Krinkle updated the task description. (Show Details)
Krinkle added a subscriber: Krinkle.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 19 2015, 5:14 PM
ori triaged this task as Normal priority.Oct 19 2015, 6:11 PM
ori set Security to None.
ori moved this task from Inbox to Backlog: Small & Maintenance on the Performance-Team board.
Krinkle added a comment.EditedDec 10 2015, 4:22 PM

Added action items to the task descriptions.

Change 258163 had a related patch set uploaded (by Krinkle):
[WIP] Add $wgObjectCacheGroups and ObjectCache::getGroup()

https://gerrit.wikimedia.org/r/258163

Change 258163 abandoned by Krinkle:
[WIP] Add $wgObjectCacheGroups and ObjectCache::getGroup()

Reason:
Closing for now. Task remains open for further exploration. Commit can be re-used later maybe.

https://gerrit.wikimedia.org/r/258163

Krinkle updated the task description. (Show Details)Mar 14 2017, 1:12 AM
Krinkle updated the task description. (Show Details)Mar 14 2017, 1:15 AM
Krinkle updated the task description. (Show Details)Mar 15 2017, 2:38 AM
Krinkle updated the task description. (Show Details)Jul 1 2017, 4:37 AM
Krinkle updated the task description. (Show Details)Sep 13 2017, 12:45 PM
Krinkle updated the task description. (Show Details)
Imarlier assigned this task to Krinkle.Jan 18 2018, 4:48 PM
Imarlier moved this task from Backlog: Small & Maintenance to Doing on the Performance-Team board.
Imarlier added a subscriber: Imarlier.

Please break this up into more discrete tasks.

I've spawned off the two unrelated pieces from this task, and repurposed this just about settling on a consistent access pattern for object cache (and deprecate the rest), and about considering to make cache groups more generalised (similar to our log groups).

Krinkle removed Krinkle as the assignee of this task.Feb 7 2018, 1:39 AM
Anomie added a subscriber: Anomie.Feb 7 2018, 2:55 PM

It seems to me that one useful improvement to the caching situation would be for there to be one interface rather than two similar but incompatible interfaces (WANObjectCache and BagOStuff).

cicalese lowered the priority of this task from Normal to Low.Feb 20 2018, 2:14 AM
cicalese moved this task from Inbox to Backlog on the MediaWiki-Platform-Team board.

Change 452870 had a related patch set uploaded (by Krinkle; owner: Legoktm):
[mediawiki/core@master] Deprecate wfGetCache() and wfGetMainCache()

https://gerrit.wikimedia.org/r/452870

Change 452870 merged by jenkins-bot:
[mediawiki/core@master] Deprecate wfGetCache() and wfGetMainCache()

https://gerrit.wikimedia.org/r/452870

Krinkle updated the task description. (Show Details)May 29 2019, 3:22 PM