Page MenuHomePhabricator
Create Task
Maniphest T327920

March 2023 Datacenter Switchover
Closed, ResolvedPublic
Actions

Description

This is the meta task for the March 2023 Datacenter switchover (eqiad -> codfw).

Switchover

Schedule

Services: Tuesday, February 28th, 2023 14:00 UTC
Traffic: Tuesday, February 28th, 2023 15:00 UTC
MediaWiki: Wednesday, March 1st, 2023 14:00 UTC

Repooling

Traffic repooling of eqiad: Wednesday, March 8th, 2023
restbase-async eqiad pooling: Wednesday, March 8th, 2023
Services/mediawiki-RO eqiad pooling: Tuesday, March 14th, 2023

Checklist

See also:

Switchback

Services: Tuesday, April 25th, 2023 14:00 UTC
Switching back: Wednesday, April 26th, 2023 14:00 UTC

Checklist

Details

SubjectRepoBranchLines +/-
wmnet: Update maintenance.eqiad.wmnet to point to mwmaint2002operations/dnsmaster+1 -1
wmnet: Update parsercache CNAMEoperations/dnsmaster+3 -3
wmnet: Update parsercache CNAMEoperations/dnsmaster+3 -3
db: Switch dns master alias to eqiadoperations/dnsmaster+15 -15
debug.json: List primary DC servers firstoperations/mediawiki-configmaster+2 -2
sre.switchdc.mediawiki: Add mw-api-int to mediawiki servicesoperations/cookbooksmaster+2 -2
wmnet: Update pcX DNSoperations/dnsmaster+3 -3
db: Switch dns master alias to codfwoperations/dnsmaster+15 -15
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedClement_GoubertT327920 March 2023 Datacenter Switchover
 ResolvedTrizek-WMFT328287 CommRel support for March 2023 Datacenter Switchover
 ResolvedClement_GoubertT328770 March 2023 Datacenter Switchover Blockers
 ResolvedRLazarusT285803 switchdc should automatically downtime "Read only" checks on DB masters being switched
ResolvedClement_GoubertT285711 Figure out plan for restbase-async w/r database switchover
 ResolvedClement_GoubertT290919 switchdc services cookbook should allow pooling services in both DCs (active/active)
 ResolvedClement_GoubertT328768 Wikitech issues for datacentre switchover (March 2023)
 ResolvedClement_GoubertT329020 Remove outdated invert-redis-sessions cookbook
 ResolvedClement_GoubertT327924 Prepare mw-on-k8s for March 2023 Datacenter Switchover
 ResolvedClement_GoubertT329065 Ensure sre.switchdc.mediawiki live test multi-DC compatibility
 ResolvedBUG REPORTClement_GoubertT329319 What should happen to Toolhub during the 2023 DC switch?
 ResolvedClement_GoubertT329533 spicerack.mysql_legacy errors on get_core_masters_heartbeats when checking x2
 ResolvedClement_GoubertT328903 March 2023 Datacenter Switchover eqiad pooling schedule
 ResolvedClement_GoubertT330651 28 February 2023 Service Switchover checklist
 ResolvedClement_GoubertT331285 March 2023 Traffic Repool checklist
 OpenClement_GoubertT328907 Post March 2023 Datacenter Switchover Tasks
 OpenNoneT328908 Migrate sre.switchdc.mediawiki to spicerack class API
 OpenNoneT328911 Expose hosts from MysqlLegacyRemoteHosts in spicerack
 OpenNoneT237773 Move Wikitech onto the production MW cluster
 Resolved• MarosteguiT167973 Move database for wikitech (labswiki) to a main cluster section
 ResolvedAndrewT188029 Move labswiki database to m5
Resolved• MarosteguiT282074 Audit labswiki grants
 ResolvedAndrewT282209 Enable connection from labweb1001/labweb1002 to s6
 DeclinedAndrewT282573 Move labweb/cloudweb hosts to private IPs
 ResolvedAndrewT284106 Replicate & sanitize wikitech data
 ResolvedAndrewT284108 Bring labswiki database tables up to date
 ResolvedLadsgroupT269348 wikitech database has almost all of its varbinary fields wrong
 DuplicateNoneT284736 Create views on labswiki on clouddb* hosts
 Resolved• BstormT287442 Create views for labswiki (wikitech)
 DeclinedNoneT237889 Install php-ldap on all MW appservers
 ResolvedtaaviT237890 Remove and empty useless user groups
 ResolvedJdforrester-WMFT196466 Remove 'shell user' right on wikitech
 DeclinedNoneT246405 On wikitech, make sysops unable to edit site JS, and then retire contentadmin
 OpenjijikiT292707 Migrate Wikitech to Kubernetes
 OpenNoneT330304 Globalize mwconfig ReadOnly
 ResolvedjbondT330985 apt.wikimedia.org post-switchover
 OpenNoneT330996 Add scap lock/unlock steps to sre.switchdc.mediawiki cookbook
 OpenNoneT330997 Support locking cookbooks run except for switchover related cookbooks
 ResolvedBUG REPORTClement_GoubertT335341 sre.discovery.datacenter breaks on services not in "production" state
 In ProgressClement_GoubertT335364 sre.discovery.datacenter should support switching the active/passive services to the other datacenter
 OpenNoneT335409 Investigate failed maintenance jobs discovered during DC switchback
 ResolvedClement_GoubertT329042 March 2023 Datacenter Switchover SRE-side Communication
 ResolvedClement_GoubertT329193 March 2023 Datacenter Switchover Excluded services
 ResolvedeoghanT329930 Switchover gitlab-replica (gitlab2002 -> gitlab1003)
 ResolvedeoghanT329931 Switchover gitlab (gitlab1004 -> gitlab2002)
 ResolvedjbondT330744 gitlab backup timer failing
 ResolvedDzahnT330091 Switchover People and Planet services to codfw
 ResolvedDzahnT330090 Switchover static miscweb services to codfw
 ResolvedClement_GoubertT330271 March 2023 Datacenter Switchover live test
 ResolvedClement_GoubertT330300 sre.switchdc.mediawiki.07-set-readwrite doesn't reset both datacenter to rw
 ResolvedClement_GoubertT330302 sre.switchdc.mediawiki.03-set-db-readonly fails in live-test mode
 Resolved• MarosteguiT330619 Enable DB replication codfw -> eqiad before the switchover
 ResolvedClement_GoubertT330650 28 February 2023 Traffic Switchover checklist
 DeclinedDzahnT330960 switch releases.wikimedia.org from eqiad to codfw
 ResolvedayounsiT330165 eqiad row B switches upgrade
 Invalid• MarosteguiT330977 Move db1183 to m1
 Resolved• MarosteguiT331510 Switchover m1 master (db1164 -> db1101)
 Resolved• MarosteguiT331511 Move db1101 to m1
 Resolved• MarosteguiT330847 Switchover m5 master (db1183 -> db1176)
 ResolvedClement_GoubertT331634 switch noc.wikimedia.org from eqiad to codfw
 ResolvedClement_GoubertT331541 14 March 2023 eqiad Service repooling
 ResolvedClement_GoubertT331981 Warmup script does not warm memcached enough
 OpenNoneT332009 Cookbooks that do DNS discovery change should check recdns
 ResolvedcmooneyT333377 eqiad row D switches upgrade
 ResolvedcmooneyT329073 eqiad row A switches upgrade
 Resolved• MarosteguiT329141 Switchover m3 master db1164 -> db1159
 Resolved• MarosteguiT329143 Move db1164 to m1
 Resolved• MarosteguiT329259 Switchover m1 master (db1176 -> db1164)
 Resolved• MarosteguiT331382 Move db1101 to m3 and make it master
 Resolved• MarosteguiT331384 Switchover m3 master db1159 -> db1101
 Resolved• MarosteguiT331387 Switchover m3 master db1101 -> db1159
 Resolved• MarosteguiT331626 Mailman hasn't delivered emails since 2023-03-07 14 UTC (was: reviewer-bot is not working)
 ResolvedayounsiT331882 eqiad row C switches upgrade
 ResolvedLadsgroupT333123 Switchover m1 master (db1101 -> db1164)
 ResolvedherronT333478 failover alert1001 to alert2001
 ResolvedherronT333837 failover alert2001 to alert1001
 DeclinedherronT333838 alerting_host: Reduced availability for job icinga-am after failover event
 OpenherronT333855 vopsbot needed manual restart after alerting hosts failover
 ResolvedJclark-ctrT308339 eqiad: move non WMCS servers out of rack C8
 ResolvedJclark-ctrT306162 Decommission mw13[07-48]
 ResolvedJoeT313327 Put mw14[57-98] in production
 Resolved• CmjohnsonT306121 Q4: (Need By: TBD) rack/setup/install mw14[57-98]
 ResolvedJclark-ctrT349587 deploy1002 lost connectivity
 ResolvedRequestJclark-ctrT350141 decommission db1131.eqiad.wmnet
 ResolvedsgrabarczukT334671 CommRel support for April 2023 Datacenter Switchback
 ResolvedClement_GoubertT335015 25 April 2023 Service Switchback checklist
 Resolved• MarosteguiT335266 Enable DB replication eqiad -> codfw before the switchover
 Resolved• MarosteguiT335267 Disconnect replication codfw -> eqiad

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Clement_Goubert updated the task description. (Show Details)Feb 28 2023, 11:32 AM
akosiaris added a comment.Feb 28 2023, 11:54 AM
In T327920#8647481, @Tgr wrote:

MwHttpRequest (that is, Guzzle/php-curl) and the URLs from https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews. I don't know if RESTBase is involved in that in some way. If you mean the VirtualRESTServiceClient in MediaWiki, that's not used (so there is no parallelism). See WikimediaPageViewService for the code.

Those URLs are RESTBase alright. And if the are being used as is from that page, that is.

  • e.g. GET https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/en.wikipedia/all-access/all-agents/Albert_Einstein/daily/2015100100/2015103100

vs the internal service-mesh

  • e.g. http://localhost:6011/wikimedia.org/v1/metrics/pageviews/per-article/en.wikipedia/all-access/all-agents/Albert_Einstein/daily/2015100100/2015103100

then MediaWiki goes via the edge caches, incurring some extra latency since there are another 3 hops before getting the actual serving cluster. In any case, something to fix after the switchover, thanks for the heads up, we 'll keep it in mind in case we have big issues.

Clement_Goubert closed subtask T330650: 28 February 2023 Traffic Switchover checklist as Resolved.Feb 28 2023, 4:41 PM
Clement_Goubert closed subtask T330651: 28 February 2023 Service Switchover checklist as Resolved.
Dzahn closed subtask T330090: Switchover static miscweb services to codfw as Resolved.Mar 1 2023, 2:11 AM
Clement_Goubert updated the task description. (Show Details)Mar 1 2023, 11:35 AM
Clement_Goubert updated the task description. (Show Details)Mar 1 2023, 12:00 PM
Stashbot added a comment.Mar 1 2023, 1:10 PM

Mentioned in SAL (#wikimedia-operations) [2023-03-01T13:10:04Z] <claime> Adding scheduled maintenance for switchover to statuspage - T327920

Clement_Goubert updated the task description. (Show Details)Mar 1 2023, 1:11 PM
Clement_Goubert updated the task description. (Show Details)
Stashbot added a comment.Mar 1 2023, 1:31 PM

Mentioned in SAL (#wikimedia-operations) [2023-03-01T13:31:04Z] <claime> Locking scap deployments for datacenter switchover - T327920

Clement_Goubert updated the task description. (Show Details)Mar 1 2023, 1:33 PM
Stashbot added a comment.Mar 1 2023, 1:40 PM

Mentioned in SAL (#wikimedia-operations) [2023-03-01T13:40:10Z] <claime> Starting mediawiki datacenter switchover step 0 - T327920

gerritbot added a comment.Mar 1 2023, 2:08 PM

Change 891552 merged by Clément Goubert:

[operations/dns@master] db: Switch dns master alias to codfw

https://gerrit.wikimedia.org/r/891552

Stashbot added a comment.Mar 1 2023, 2:08 PM

Mentioned in SAL (#wikimedia-operations) [2023-03-01T14:08:18Z] <claime> Phase 9.5 Update DNS records for new database masters - T327920

Stashbot added a comment.Mar 1 2023, 2:09 PM

Mentioned in SAL (#wikimedia-operations) [2023-03-01T14:09:56Z] <claime> Phase 9.5 DNS records for new database masters updated - T327920

Stashbot added a comment.Mar 1 2023, 2:16 PM

Mentioned in SAL (#wikimedia-operations) [2023-03-01T14:16:04Z] <claime> Removing scap lock - T327920

gerritbot added a comment.Mar 1 2023, 2:16 PM

Change 893479 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/dns@master] wmnet: Update pcX DNS

https://gerrit.wikimedia.org/r/893479

gerritbot added a comment.Mar 1 2023, 2:17 PM

Change 892428 merged by jenkins-bot:

[operations/mediawiki-config@master] debug.json: List primary DC servers first

https://gerrit.wikimedia.org/r/892428

Stashbot added a comment.Mar 1 2023, 2:18 PM

Mentioned in SAL (#wikimedia-operations) [2023-03-01T14:18:24Z] <cgoubert@deploy2002> Started scap: Backport for [[gerrit:892428|debug.json: List primary DC servers first (T327920)]]

Stashbot added a comment.Mar 1 2023, 2:20 PM

Mentioned in SAL (#wikimedia-operations) [2023-03-01T14:20:30Z] <cgoubert@deploy2002> cgoubert: Backport for [[gerrit:892428|debug.json: List primary DC servers first (T327920)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet

gerritbot added a comment.Mar 1 2023, 2:23 PM

Change 893479 merged by Marostegui:

[operations/dns@master] wmnet: Update pcX DNS

https://gerrit.wikimedia.org/r/893479

Stashbot added a comment.Mar 1 2023, 2:26 PM

Mentioned in SAL (#wikimedia-operations) [2023-03-01T14:26:18Z] <cgoubert@deploy2002> Finished scap: Backport for [[gerrit:892428|debug.json: List primary DC servers first (T327920)]] (duration: 07m 54s)

Clement_Goubert updated the task description. (Show Details)Mar 1 2023, 2:26 PM
Stashbot added a comment.Mar 1 2023, 2:27 PM

Mentioned in SAL (#wikimedia-operations) [2023-03-01T14:27:02Z] <claime> End mediawiki datacenter switchover - T327920

Dzahn added a subtask: T330960: switch releases.wikimedia.org from eqiad to codfw.Mar 2 2023, 1:02 AM
Dzahn added a subtask: T330963: switch doc.wikimedia.org from eqiad to codfw.Mar 2 2023, 1:18 AM
gerritbot added a comment.Mar 2 2023, 10:09 AM

Change 893675 had a related patch set uploaded (by Zabe; author: Zabe):

[operations/dns@master] wmnet: Update maintenance.eqiad.wmnet to point to mwmaint2002

https://gerrit.wikimedia.org/r/893675

gerritbot added a comment.Mar 2 2023, 10:41 AM

Change 893675 merged by Clément Goubert:

[operations/dns@master] wmnet: Update maintenance.eqiad.wmnet to point to mwmaint2002

https://gerrit.wikimedia.org/r/893675

Dzahn subscribed.Mar 3 2023, 2:05 AM

https://releases.wikimedia.org switched to codfw (T330960)

Dzahn changed the status of subtask T330960: switch releases.wikimedia.org from eqiad to codfw from Open to Stalled.Mar 3 2023, 3:12 PM
Tgr added a comment.Mar 4 2023, 2:20 AM
In T327920#8652181, @akosiaris wrote:

And if the are being used as is from that page, that is.

  • e.g. GET https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/en.wikipedia/all-access/all-agents/Albert_Einstein/daily/2015100100/2015103100

vs the internal service-mesh

  • e.g. http://localhost:6011/wikimedia.org/v1/metrics/pageviews/per-article/en.wikipedia/all-access/all-agents/Albert_Einstein/daily/2015100100/2015103100

then MediaWiki goes via the edge caches, incurring some extra latency since there are another 3 hops before getting the actual serving cluster.

Yeah, it uses the public URL. Ping me when it is a good time to fix that, it should be trivial.

Clement_Goubert updated the task description. (Show Details)Mar 6 2023, 11:57 AM
Clement_Goubert added a subtask: T331285: March 2023 Traffic Repool checklist.Mar 6 2023, 12:05 PM
eoghan closed subtask T329931: Switchover gitlab (gitlab1004 -> gitlab2002) as Resolved.Mar 6 2023, 2:57 PM
Dzahn reopened subtask T330091: Switchover People and Planet services to codfw as In Progress.Mar 6 2023, 11:28 PM
Dzahn closed subtask T330091: Switchover People and Planet services to codfw as Resolved.Mar 7 2023, 12:13 AM
Clement_Goubert added a subtask: T330165: eqiad row B switches upgrade.Mar 8 2023, 10:02 AM
Clement_Goubert closed subtask T331285: March 2023 Traffic Repool checklist as Resolved.Mar 8 2023, 11:45 AM
Clement_Goubert reopened subtask T330651: 28 February 2023 Service Switchover checklist as In Progress.
Clement_Goubert closed subtask T330651: 28 February 2023 Service Switchover checklist as Resolved.Mar 8 2023, 12:02 PM
Clement_Goubert closed subtask T328903: March 2023 Datacenter Switchover eqiad pooling schedule as Resolved.
Clement_Goubert reopened subtask T328903: March 2023 Datacenter Switchover eqiad pooling schedule as In Progress.Mar 8 2023, 2:23 PM
Kappakayala subscribed.Mar 8 2023, 2:50 PM
Clement_Goubert updated the task description. (Show Details)Mar 8 2023, 3:20 PM
Clement_Goubert changed the status of subtask T331634: switch noc.wikimedia.org from eqiad to codfw from Open to In Progress.Mar 13 2023, 2:21 PM
Clement_Goubert closed subtask T331634: switch noc.wikimedia.org from eqiad to codfw as Resolved.Mar 13 2023, 2:25 PM
Clement_Goubert added a subtask: T331541: 14 March 2023 eqiad Service repooling.Mar 14 2023, 11:38 AM
Clement_Goubert closed subtask T331541: 14 March 2023 eqiad Service repooling as Resolved.Mar 14 2023, 2:19 PM
Dzahn closed subtask T330960: switch releases.wikimedia.org from eqiad to codfw as Declined.Mar 14 2023, 6:23 PM
Dzahn removed a subtask: T330963: switch doc.wikimedia.org from eqiad to codfw.
Trizek-WMF closed subtask T328287: CommRel support for March 2023 Datacenter Switchover as Resolved.Mar 27 2023, 2:53 PM
ayounsi closed subtask T330165: eqiad row B switches upgrade as Resolved.Mar 29 2023, 7:35 AM
ayounsi edited subtasks, added: T333377: eqiad row D switches upgrade, T329073: eqiad row A switches upgrade, T331882: eqiad row C switches upgrade, T308339: eqiad: move non WMCS servers out of rack C8; removed: T327248: eqiad/codfw virtual-chassis upgrades.Apr 12 2023, 12:30 PM
cmooney closed subtask T333377: eqiad row D switches upgrade as Resolved.Apr 18 2023, 5:02 PM
Clement_Goubert updated the task description. (Show Details)Apr 19 2023, 8:03 AM
• Marostegui updated the task description. (Show Details)Apr 19 2023, 8:10 AM
Trizek-WMF changed the status of subtask T334671: CommRel support for April 2023 Datacenter Switchback from Open to In Progress.Apr 19 2023, 5:49 PM
kostajh mentioned this in T335125: Account creation attempt on mobile Wikipedia domain leads user to desktop Special:CentralLogin/complete, often in logged-out state.Apr 20 2023, 12:37 PM
Clement_Goubert updated the task description. (Show Details)Apr 24 2023, 8:48 AM
Stashbot added a comment.Apr 24 2023, 10:27 AM

Mentioned in SAL (#wikimedia-operations) [2023-04-24T10:27:40Z] <claime> Datacenter switchover live testing setting db to read-only and back in eqiad - T327920

Stashbot added a comment.Apr 24 2023, 10:29 AM

Mentioned in SAL (#wikimedia-operations) [2023-04-24T10:29:58Z] <claime> Datacenter switchover live testing setting db to read-only and back in eqiad successful - T327920

• Marostegui closed subtask T335266: Enable DB replication eqiad -> codfw before the switchover as Resolved.Apr 25 2023, 6:12 AM
gerritbot added a comment.Apr 25 2023, 10:42 AM

Change 911780 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/cookbooks@master] sre.switchdc.mediawiki: Add mw-api-int to mediawiki services

https://gerrit.wikimedia.org/r/911780

gerritbot added a comment.Apr 25 2023, 10:48 AM

Change 911780 merged by jenkins-bot:

[operations/cookbooks@master] sre.switchdc.mediawiki: Add mw-api-int to mediawiki services

https://gerrit.wikimedia.org/r/911780

Clement_Goubert changed the status of subtask T335015: 25 April 2023 Service Switchback checklist from Open to In Progress.Apr 25 2023, 1:33 PM
Clement_Goubert closed subtask T335015: 25 April 2023 Service Switchback checklist as Resolved.Apr 25 2023, 3:22 PM
Clement_Goubert closed subtask T328903: March 2023 Datacenter Switchover eqiad pooling schedule as Resolved.Apr 25 2023, 3:41 PM
Clement_Goubert closed subtask T329193: March 2023 Datacenter Switchover Excluded services as Resolved.
gerritbot added a comment.Apr 26 2023, 5:53 AM

Change 912171 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/dns@master] wmnet: Update parsercache CNAME

https://gerrit.wikimedia.org/r/912171

• Marostegui updated the task description. (Show Details)Apr 26 2023, 5:53 AM
gerritbot added a comment.Apr 26 2023, 8:40 AM

Change 912235 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/dns@master] db: Switch dns master alias to eqiad

https://gerrit.wikimedia.org/r/912235

Clement_Goubert updated the task description. (Show Details)Apr 26 2023, 8:41 AM
Clement_Goubert updated the task description. (Show Details)
Stashbot added a comment.Apr 26 2023, 1:14 PM

Mentioned in SAL (#wikimedia-operations) [2023-04-26T13:13:59Z] <claime> Locking scap for datacenter switchback - T327920

Stashbot added a comment.Apr 26 2023, 1:15 PM

Mentioned in SAL (#wikimedia-operations) [2023-04-26T13:15:43Z] <cgoubert@deploy1002> Locking from deployment [ALL REPOSITORIES]: Datacenter Switchback - T327920

Clement_Goubert updated the task description. (Show Details)Apr 26 2023, 1:16 PM
Stashbot added a comment.Apr 26 2023, 1:23 PM

Mentioned in SAL (#wikimedia-operations) [2023-04-26T13:23:27Z] <claime> Starting mediawiki datacenter switchback preparation - T327920

Clement_Goubert updated the task description. (Show Details)Apr 26 2023, 1:28 PM
Clement_Goubert updated the task description. (Show Details)Apr 26 2023, 1:38 PM
Stashbot added a comment.Apr 26 2023, 1:45 PM

Mentioned in SAL (#wikimedia-operations) [2023-04-26T13:45:16Z] <claime> Stopping maintenance scripts for datacenter switchback - T327920

Stashbot added a comment.Apr 26 2023, 1:59 PM

Mentioned in SAL (#wikimedia-operations) [2023-04-26T13:59:49Z] <claime> Going to read-only for mediawiki datacenter switchback - T327920

Stashbot added a comment.Apr 26 2023, 2:05 PM

Mentioned in SAL (#wikimedia-operations) [2023-04-26T14:05:00Z] <claime> Restarting maintenance jobs - T327920

gerritbot added a comment.Apr 26 2023, 2:08 PM

Change 912235 merged by Clément Goubert:

[operations/dns@master] db: Switch dns master alias to eqiad

https://gerrit.wikimedia.org/r/912235

Clement_Goubert updated the task description. (Show Details)Apr 26 2023, 2:13 PM
gerritbot added a comment.Apr 26 2023, 2:14 PM

Change 912302 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/dns@master] wmnet: Update parsercache CNAME

https://gerrit.wikimedia.org/r/912302

gerritbot added a comment.Apr 26 2023, 2:14 PM

Change 912171 abandoned by Marostegui:

[operations/dns@master] wmnet: Update parsercache CNAME

Reason:

I messed up the rebase, so abandoning in favour of 912302

https://gerrit.wikimedia.org/r/912171

RLazarus added a subtask: T335409: Investigate failed maintenance jobs discovered during DC switchback.Apr 26 2023, 2:15 PM
gerritbot added a comment.Apr 26 2023, 2:15 PM

Change 912302 merged by Marostegui:

[operations/dns@master] wmnet: Update parsercache CNAME

https://gerrit.wikimedia.org/r/912302

Stashbot added a comment.Apr 26 2023, 2:16 PM

Mentioned in SAL (#wikimedia-operations) [2023-04-26T14:16:00Z] <marostegui> Update dns for parsercache T327920

• Marostegui updated the task description. (Show Details)Apr 26 2023, 2:16 PM
Stashbot added a comment.Apr 26 2023, 2:24 PM

Mentioned in SAL (#wikimedia-operations) [2023-04-26T14:24:46Z] <cgoubert@deploy1002> Unlocked for deployment [ALL REPOSITORIES]: Datacenter Switchback - T327920 (duration: 69m 03s)

Clement_Goubert closed this task as Resolved.Apr 26 2023, 2:37 PM
Clement_Goubert updated the task description. (Show Details)
Ladsgroup awarded a token.Apr 26 2023, 2:39 PM
Clement_Goubert closed subtask T329042: March 2023 Datacenter Switchover SRE-side Communication as Resolved.Apr 26 2023, 2:46 PM
Trizek-WMF closed subtask T334671: CommRel support for April 2023 Datacenter Switchback as Resolved.May 30 2023, 3:45 PM
Jclark-ctr closed subtask T308339: eqiad: move non WMCS servers out of rack C8 as Resolved.Nov 1 2023, 5:30 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL