Page MenuHomePhabricator

MoveComms support for Southward Datacentre Switchover (Sept. 2026)
Closed, ResolvedPublic

Description

Dear MoveComms,

We are planning a datacentre switchover for the week of September 21st (week 39) with the following schedule:

The expected impact is 2-3 minutes of read-only on Wednesday, 23 September 2026 @ 14:00 UTC.

We continue to follow the process described in Recurring, Equinox-based, Data Center Switchovers, in particular:

  • There is no switchback! We are staying in codfw until the next switchover.
  • Future switchovers are predictable and take place every 6 months; always on the week of an equinox.

Let ServiceOps know if you need more info on the changes.

Thank you!

Planning

As soon as the task is received by Movement Communications

Three weeks before

  • Tech News message (initial warning in this week's issue, reuse this message)
  • Update the message to communities
    • Check on dates and links
    • Edit qqq messages with the new Zonestamp link
  • Have the message being translated by emailing both:
    • translators-l for translations
    • wikitech-ambassadors for information and translations
  • Monitor the message's talk page

Two weeks before

  • Central Notice Banner
  • Tech News ("this is happening next week" in this week's issue)

The week before

  • Tech News ("this is happening this week" in this week's issue)
  • Agree with SRE on who will inform Movement Communication of the advancement and success of the process.

The week it happens

  • monitor the wikis
  • when the read-only is done and confirmed by SRE's spokeperson, remove the banner.

The week after, maximum

  • Debrief how it went, and document (comment on this task)
  • Archive the campaign

The Read-only is reported

Step 1
Once SRE informs MovComms of the report, ask:

  • where is the public documentation
  • what can be communicated, in a plain-humain language
  • if a new date is known

Step 2

  • De-activate the the campaign.
  • Edit Meta's main page calendar to indicate that the read-only is postponed.
  • Inform the following places, with the reason and the new date (if known):

Details

Other Assignee
IKristiani-WMF

Event Timeline

Trizek-WMF subscribed.

Hi @SLyngshede-WMF! I'll be your host for this switch.

Has anything major changed since the last switch?

@Trizek-WMF Thank you. No the switch over will be "business as usual".

Trizek-WMF changed the task status from Open to In Progress.Sep 4 2026, 2:25 PM
Trizek-WMF triaged this task as High priority.
Trizek-WMF updated the task description. (Show Details)
Trizek-WMF updated the task description. (Show Details)

Two points for you @SLyngshede-WMF:

  • SRE informs WMF staff using the internal channels
  • Who will inform Movement Communication of the advancement and success of the process (implying how and when)?

@Trizek-WMF I'll be handling the information. I can keep you updated on this task, unless you prefer some other channel.

@Trizek-WMF I'll be handling the information. I can keep you updated on this task, unless you prefer some other channel.

Sorry for the delay responding. Please update this task.

@Trizek-WMF Hi, first day of switchover revealed a capacity issue. We're cancelling the rest of the planned maintenance to avoid any risk, while we work on resolving the issue.

We'll keep you update as we know more, in the meantime is there anything else I can provide you to help communication?

@SLyngshede-WMF, thank you for letting me know.

As no read-only will happen today, I'll process on the checklist for cancellation: removing banners and informing our audiences. Is there any public documentation I can rely on?

Update from SRE:

  • The initial steps of the datacenter switchover exercise revealed capacity issues, preventing us from switching all services, so we decided to pause the planned switchover.
  • We’re prioritizing investigating that issue, not just to complete the switchover exercise, but to ensure that we continue to be able to serve our users reliably.
  • An updated timeline for the planned switchover will be sent out when we know more. Meanwhile, all traffic and edits continue to work as usual.
Trizek-WMF lowered the priority of this task from High to Medium.Wed, Sep 23, 12:28 PM
Trizek-WMF updated the task description. (Show Details)

Keeping it open, to track reactions. To be closed at the end of the week.

Please re-open a new task when the new read-only time is scheduled.