As a final step, we want to shift writes from Cirrus to SUP. To reduce the risk of loosing data, we take the following measures:
The batches are:
AC:
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | EBernhardson | T363475 SUP: Shift Writes from Cirrus to SUP | |||
| Resolved | EBernhardson | T365190 Cannot provide empty array to wikis as $wgCirrusSearchWriteClusters |
Change #1024411 had a related patch set uploaded (by Peter Fischer; author: Peter Fischer):
[operations/mediawiki-config@master] Shift writes to SUP, 1st batch
Change #1025411 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):
[operations/deployment-charts@master] cirrus updater: Enable codfw consumer
Change #1025411 merged by jenkins-bot:
[operations/deployment-charts@master] cirrus updater: Enable codfw consumer
Change #1024411 merged by jenkins-bot:
[operations/mediawiki-config@master] Shift writes to SUP, 1st batch
Mentioned in SAL (#wikimedia-operations) [2024-04-29T20:05:39Z] <cjming@deploy1002> Started scap: Backport for [[gerrit:1024411|Shift writes to SUP, 1st batch (T363475)]]
Mentioned in SAL (#wikimedia-operations) [2024-04-29T20:09:10Z] <cjming@deploy1002> cjming and pfischer: Backport for [[gerrit:1024411|Shift writes to SUP, 1st batch (T363475)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)
Mentioned in SAL (#wikimedia-operations) [2024-04-29T20:21:56Z] <cjming@deploy1002> Finished scap: Backport for [[gerrit:1024411|Shift writes to SUP, 1st batch (T363475)]] (duration: 16m 17s)
Change #1029273 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):
[operations/mediawiki-config@master] cirrus: Shift remaining public wikis in codfw to replacement updater
Change #1029274 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):
[operations/deployment-charts@master] cirrus: Expand codfw to serve writes to all public wikis
Change #1029274 merged by jenkins-bot:
[operations/deployment-charts@master] cirrus: Expand codfw to serve writes to all public wikis
Change #1029273 merged by jenkins-bot:
[operations/mediawiki-config@master] cirrus: Shift remaining public wikis in codfw to replacement updater
Mentioned in SAL (#wikimedia-operations) [2024-05-08T20:09:54Z] <ebernhardson@deploy1002> Started scap: Backport for [[gerrit:1029273|cirrus: Shift remaining public wikis in codfw to replacement updater (T363475)]]
Mentioned in SAL (#wikimedia-operations) [2024-05-08T20:12:34Z] <ebernhardson@deploy1002> ebernhardson: Backport for [[gerrit:1029273|cirrus: Shift remaining public wikis in codfw to replacement updater (T363475)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)
Mentioned in SAL (#wikimedia-operations) [2024-05-08T20:25:54Z] <ebernhardson@deploy1002> Finished scap: Backport for [[gerrit:1029273|cirrus: Shift remaining public wikis in codfw to replacement updater (T363475)]] (duration: 16m 00s)
Peter's on a WDQS task, Erik will take this after the Discolytics search metrics task.
Change #1031027 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):
[operations/deployment-charts@master] cirrus: Deploy updater to eqiad at 25% load
Change #1031029 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):
[operations/mediawiki-config@master] cirrus: Shift 25% of public wikis writes in eqiad to replacement updater
Change #1031027 merged by jenkins-bot:
[operations/deployment-charts@master] cirrus: Deploy updater to eqiad at 25% load
Change #1031029 merged by jenkins-bot:
[operations/mediawiki-config@master] cirrus: Shift 25% of public wikis writes in eqiad to replacement updater
Mentioned in SAL (#wikimedia-operations) [2024-05-14T20:26:22Z] <cjming@deploy1002> Started scap: Backport for [[gerrit:1031029|cirrus: Shift 25% of public wikis writes in eqiad to replacement updater (T363475)]]
Mentioned in SAL (#wikimedia-operations) [2024-05-14T20:28:58Z] <cjming@deploy1002> cjming and ebernhardson: Backport for [[gerrit:1031029|cirrus: Shift 25% of public wikis writes in eqiad to replacement updater (T363475)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)
Mentioned in SAL (#wikimedia-operations) [2024-05-14T20:41:25Z] <cjming@deploy1002> Finished scap: Backport for [[gerrit:1031029|cirrus: Shift 25% of public wikis writes in eqiad to replacement updater (T363475)]] (duration: 15m 02s)
Change #1032017 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):
[mediawiki/extensions/CirrusSearch@master] Switch DefaultWriteClusters settings to provide_default
Change #1032017 abandoned by Ebernhardson:
[mediawiki/extensions/CirrusSearch@master] Switch DefaultWriteClusters settings to provide_default
Reason:
On review of how extension registration works, this is simply not possible. With the way values are passed around, it looks like provide_default can only provide arrays. This is being fixed via
Ia3eaf6a6078 instead.
We are getting alerts from "MediaWiki CirrusSearch update rate - codfw". I've silenced them for the moment, but we might want to make sure we have the appropriate alerts for the new SUP and that the old one does not raise false positives.
Change #1036723 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):
[operations/mediawiki-config@master] cirrus: Move remaining public writes to SUP
Change #1036723 merged by jenkins-bot:
[operations/mediawiki-config@master] cirrus: Move remaining public writes to SUP
Mentioned in SAL (#wikimedia-operations) [2024-05-28T20:22:20Z] <cjming@deploy1002> Started scap: Backport for [[gerrit:1036723|cirrus: Move remaining public writes to SUP (T363475)]]
Mentioned in SAL (#wikimedia-operations) [2024-05-28T20:25:26Z] <cjming@deploy1002> cjming and ebernhardson: Backport for [[gerrit:1036723|cirrus: Move remaining public writes to SUP (T363475)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)
Mentioned in SAL (#wikimedia-operations) [2024-05-28T20:34:31Z] <cjming@deploy1002> Finished scap: Backport for [[gerrit:1036723|cirrus: Move remaining public writes to SUP (T363475)]] (duration: 12m 11s)