Page MenuHomePhabricator

Remove 'db-replicated' and ReplicatedBagOStuff
Open, Stalled, MediumPublic

Description

The ReplicatedBagOStuff was introduced in 2016 as part of Multi-DC preparation for SessionStore (T88634). The idea involved a Redis backend for session store, and a presumed requirement for SessionStore to have to be able to read from the primary DC at all times (READ_LATEST). To satisfy the latter, given the current RedisBagOStuff implementation, would require one RedisBag instance that connects to the local pool of Redis instances, and one that connects to the primary DC's pool.

Background: https://wikitech.wikimedia.org/wiki/Performance/Multi-DC_MediaWiki.

The ReplicatedBagOStuff class ended up unused [1] for a numer of reasons:

From SessionManager docs on doc.wikimedia.org:
  • Locally writable data. The data must be writable from both primary and secondary data centres.
  • Locally latest reads. Writes must by default be immediately visible […] from web servers in that data centre.
  • Replication. The data must be eventually consistent across all data centres. […]

This means it is no longer expected to be able to read from the primary DC, also no longer expected to send writes or deletes directly to the primary DC. Thus, removing the need for the two-bag wrapper.

  • (T254634 and T91820) ChronologyProtector and temporary routing cookies "stick" the browser to the primary DC during DB writes. The majority of session writes are also expected to happen via the primary DC, thus by the time a browser unsticks, replication should have completed. If use cases exist that write to the session from a secondary DC request (i.e. GET), they may not rely on immediate chronology between this and a subsequent POST request (per the above requirements).
  • SessionStore moved from Redis to Kask/Cassandra.

The db-replicated cache ID was introduced in 2015 as part of Multi-DC preparation for MainStash DB (T88493). It ended up redundant with CACHE_DB when the MainStash DB was finalised in 2022 (T212129) because:

  • The built-in SqlBagOStuff was improved to support re-using the main Rdbms-LoadBalancer object, and this is also the default now.
  • The requirements for MainStash (documented in MainSchemaConfig.php#MainStash) allow reads to be stale when performed in a secondary DC. This removed the need for direct reads from the primary DC.
  • Even at WMF scale we do not currently anticipate a need to separate reads from writes in terms of MainStash load on DBs. This removes the need for SqlBag to obtain separate write connections and read connections, and thus removes the need to configure a "replica" and "primary" instance.
    • In terms of hypothetical future needs, if load separation were needed, we still don't need ReplicatedBagOStuff, since SqlBagOStuff has built-in capabilities already both to shard across multiple servers (like for ParserCache), or to leverage $wgLBFactoryConf via Rdbms-LoadBalancer, and define a primary DB and one or more replicas, which SqlBagOStuff could then be tweaked to make use of when it obtains connections.

Scope of work

  • Remove db-replicated without deprecation (public references go via MainStash instead, unaffected).
  • Deprecate ReplicatedBagOStuff in MW 1.42.
  • Remove ReplicatedBagOStuff in MW 1.43.

Reflist

[1] https://codesearch.wmcloud.org/search/?q=ReplicatedBagOStuff&files=&excludeFiles=&repos= (no known usage in core or extensions or elsewhere).

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 981564 had a related patch set uploaded (by D3r1ck01; author: Derick Alangi):

[mediawiki/core@master] objectcache: Drop support for `$wgObjectCaches['db-replicated']`

https://gerrit.wikimedia.org/r/981564

Change 981564 merged by jenkins-bot:

[mediawiki/core@master] objectcache: Drop support for `$wgObjectCaches['db-replicated']`

https://gerrit.wikimedia.org/r/981564

Change 982358 had a related patch set uploaded (by D3r1ck01; author: Derick Alangi):

[mediawiki/core@master] objectcache: Soft deprecate ReplicatedBagOStuff

https://gerrit.wikimedia.org/r/982358

Change 982358 merged by jenkins-bot:

[mediawiki/core@master] objectcache: Soft deprecate ReplicatedBagOStuff

https://gerrit.wikimedia.org/r/982358

Krinkle changed the task status from Open to Stalled.Jan 5 2024, 2:14 PM
Krinkle triaged this task as Low priority.
Krinkle raised the priority of this task from Low to Medium.
Krinkle updated the task description. (Show Details)

Waiting for 1.43 release cycle.

Change 1012345 had a related patch set uploaded (by D3r1ck01; author: Derick Alangi):

[mediawiki/core@master] objectcache: Drop ReplicatedBagOStuff

https://gerrit.wikimedia.org/r/1012345