Page MenuHomePhabricator

Refactor BagOStuff to use a more storage/multi-DC aware interface hierarchy
Open, MediumPublic

Description

Currently, the BagOStuff class has a lot of methods that are tricky to implement outside of a single-DC cluster instance, for example incr(), lock(), add(), merge(), and so on. By splitting out some interfaces, it becomes possible to better map caching interfaces to storage mediums. For example, RESTBagOStuff would not have to have a bunch of broken "don't actually use this for this subclass" methods, but could rather implement only methods that make sense.

Proposed interfaces are:

  • KeyValueStore: Basic set/get methods. This is useful for backends that cannot implement mixed read/write methods efficiently, e.g. add(), merge(), incr(), changeTTL().
  • MergeableKeyValueStore: This defines add() and merge() methods as well as a new consume() method, which is a better an alternative to changeTTL() for token claiming (e.g. OAuth/CentralAuth).
  • AdvancedKeyValueStore: This defines incr(), decr(), incrWithInit(), changeTTL(), changeTTLMulti(), lock(), unlock(), and getScopedLock().

Event Timeline

aaron created this task.Oct 24 2019, 5:16 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 24 2019, 5:16 PM

Change 524663 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] objectcache: split out various interfaces from BagOStuff

https://gerrit.wikimedia.org/r/524663

Krinkle added a subscriber: Joe.EditedOct 24 2019, 5:30 PM
Krinkle added a subscriber: Krinkle.

@Joe These are the BagOStuff interfaces I was referring to yesterday.

daniel triaged this task as Low priority.Nov 11 2019, 8:01 PM
daniel raised the priority of this task from Low to Medium.Nov 11 2019, 8:11 PM
daniel added a subscriber: daniel.

Probably needs discussion and re-architecting. Note that php interfaces cannot really be refactored, they can only be replaced.
This doesn't seem immediately actionable, and thus unsuitable as a clinic duty ticket. May make more sense to track this as a future initiative, or an rfc.

kchapman added a subscriber: kchapman.

@daniel how big a project is re-architecting this? I moved to future initiative

@daniel how big a project is re-architecting this? I moved to future initiative

Medium sized. Depends on how much effort we spend surveying current usages and patterns.

Aklapper renamed this task from Refactor BagOStuff to use a more storage/multi-DC aware interface hieararchy to Refactor BagOStuff to use a more storage/multi-DC aware interface hierarchy.Nov 13 2019, 11:46 PM

Change 563289 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] objectcache: split out StorageAwareness/ExpirationAwareness from IExpiringStore

https://gerrit.wikimedia.org/r/563289

Change 563289 merged by jenkins-bot:
[mediawiki/core@master] objectcache: split out StorageAwareness/ExpirationAwareness from IExpiringStore

https://gerrit.wikimedia.org/r/563289