Page MenuHomePhabricator

Reduce and clarify ObjectCache entry points
Open, LowPublic

Description

We have too many different cache objects, configuration variables, methods and factories for various types of cache interfaces.

Entry points

A few of the things we use right now:

  • wfGetCache( "hash" ), new HashBagOStuff();.
  • wfGetCache( CACHE_ANYTHING ), ObjectCache::newAnything().
  • ObjectCache::newAccelerator( fallback ).
  • wfGetMainCache(), ObjectCache::getMainClusterInstance().
  • ObjectCache::getMainWANInstance().
  • ObjectCache::getMainStashInstance().
  • ObjectCache::getInstance().
  • ObjectCache::getWANInstance().
  • more..
  • variations on the above due to back-compat aliases.

We should reduce this to a handful only that we use everywhere.

With the multi-datacenter work (T88445) we're also changing which cache developers should use by default. Namely one should now default to WANCache instead of the local-dc cache. It may make sense to also update the notion of "main cache" to start referring to WAN cache to clarify this convention shift.

I propose we standardise on the following four main entry points:

  • Server cache or Process cache (e.g. APC, fallback to none, or hash.)
  • Local cluster cache (e.g. Memcached).
  • WAN cache (e.g. Memcached, with relayed purges and other improvements).
  • Main stash (e.g. db-replicated).

With two extra entry points for special cases:

  • Identified cache group (e.g. ObjectCache::getGroup( "language-converter" )).
  • Identified cache backend (should be unused, but for back-compat, e.g. ObjectCache::getInstance( CACHE_DB );

Reduced interface:

class ObjectCache {
 static function getInstance( string $type ); // Key from $wgObjectCaches

 static function getGroup( string $groupName ); // Key from $wgObjectCacheGroups, defaults to main cache

 static function newAccelerator();

 static function getLocalClusterCache();

 static function getMainCache();

 static function getMainStash();
}

Action items:

  • Don't require custom accelerator fallback. Set a default one (empty or hash) and deprecate the fallback parameter.
  • Give CACHE_ACCEL sensible default so it doesn't fail on plain installs.
  • Reduce ObjectCache entry points to just WAN, LocalServer (APC), LocalCluster (Main) and stash.
  • Deprecate wfGetCache(), wfGetMainCache().

Configuration

Some configuration:

  • $wgObjectCaches array( .. )
  • $wgWANObjectCaches array( .. )
  • $wgMainCacheType
  • $wgMainWANCache
  • $wgMainStash
  • $wgMessageCacheType, $wgParserCacheType, $wgSessionCacheType, $wgLanguageConverterCacheType (maybe introduce some kind of cache grouping to allow overriding backends for individual cache groups, defaulting to main cache).

Action items:

  • Deprecate cache group config vars ($wgMessageCacheType, $wgParserCacheType etc.) in favour of e.g. $wgObjectCacheGroups.
  • Implement ObjectCache::getGroup( $name ) with default to wgMainCacheType.
  • Deprecate wfGetMessageCacheStorage(), wfGetParserCacheStorage()

Once we have $wgObjectCacheGroups, it'll be less complicated to introduce new cache groups. Making it easy to move around things in wmf-production and re-use caches without having to add more wg*CacheType variables and ad-hoc instantiation of BagOStuff objects.

See also:

Event Timeline

Krinkle raised the priority of this task from to Needs Triage.
Krinkle updated the task description. (Show Details)
Krinkle added a subscriber: Krinkle.
ori triaged this task as Medium priority.Oct 19 2015, 6:11 PM
ori set Security to None.
ori moved this task from Inbox to Backlog: Maintenance on the Performance-Team board.

Added action items to the task descriptions.

Change 258163 had a related patch set uploaded (by Krinkle):
[WIP] Add $wgObjectCacheGroups and ObjectCache::getGroup()

https://gerrit.wikimedia.org/r/258163

Change 258163 abandoned by Krinkle:
[WIP] Add $wgObjectCacheGroups and ObjectCache::getGroup()

Reason:
Closing for now. Task remains open for further exploration. Commit can be re-used later maybe.

https://gerrit.wikimedia.org/r/258163

Krinkle updated the task description. (Show Details)
Imarlier moved this task from Backlog: Maintenance to Doing (old) on the Performance-Team board.
Imarlier added a subscriber: Imarlier.

Please break this up into more discrete tasks.

I've spawned off the two unrelated pieces from this task, and repurposed this just about settling on a consistent access pattern for object cache (and deprecate the rest), and about considering to make cache groups more generalised (similar to our log groups).

It seems to me that one useful improvement to the caching situation would be for there to be one interface rather than two similar but incompatible interfaces (WANObjectCache and BagOStuff).

cicalese lowered the priority of this task from Medium to Low.Feb 20 2018, 2:14 AM
cicalese moved this task from Inbox to Backlog on the MediaWiki-Platform-Team board.

Change 452870 had a related patch set uploaded (by Krinkle; owner: Legoktm):
[mediawiki/core@master] Deprecate wfGetCache() and wfGetMainCache()

https://gerrit.wikimedia.org/r/452870

Change 452870 merged by jenkins-bot:
[mediawiki/core@master] Deprecate wfGetCache() and wfGetMainCache()

https://gerrit.wikimedia.org/r/452870