Page MenuHomePhabricator

Community Relations support for the 2018 data center switchover
Closed, ResolvedPublic

Description

What is the problem?

The Site Reliability Team, with the assistance of many other affected teams in the organization, will be performing a data center switchover from our primary data center (eqiad) to our secondary data center (codfw) for MediaWiki and other associated services, and a switch back to the original state, 3-4 weeks later.

The problem is that this is user-visible, as all wikis will need to be made read-only on both occasions for a period of roughly 10 minutes.

How can we help you?

As in previous years, we will need to communicate the upcoming downtime for both occasions to our users/communities, and explain why we are doing this.

What does success look like?

Success would look like awareness by all regular users of our wikis that this will happen and that they won't be able to make any changes for a short period of time (twice), and, as in recent years, some understanding by our users why this maintenance is important in protecting our sites and services with disaster readiness. We were very happy with how Community Relations has handled this project in the past 2 years and are looking for a very similar result.

What is your deadline?

Tentative timeplan currently is. Note that only the mediawiki part is expected to have visible user repercussions as the wikis will need to be in a read-only mode for a while.

Switchover

Services: Tuesday, September 11th 2018 14:30 UTC
Media storage/Swift: Tuesday, September 11th 2018 15:00 UTC
Traffic: Tuesday, September 11th 2018 19:00 UTC
Mediawiki: Wednesday, September 12th 2018: 14:00 UTC

Switchback:

Traffic: Wednesday, October 10th 2018 09:00 UTC (and maybe some prep work on Monday)
Mediawiki: Wednesday, October 10th 2018: 14:00 UTC
Services: Thursday, October 11th 2018 14:30 UTC
Media storage/Swift: Thursday, October 11th 2018 15:00 UTC

We should probably be alerting at least 2-3 weeks beforehand

Event Timeline

Thanks a lot for filing this, Mark!
Johan should be able to assist then. @Whatamidoing-WMF will be his backup.

Johan triaged this task as Medium priority.Jul 18 2018, 2:57 PM

@mark @akosiaris So what's going out to Monday is in https://meta.wikimedia.org/wiki/Tech/News/2018/35 which links to https://meta.wikimedia.org/wiki/Tech/Server_switch_2018 (basically a copy of last year's message with relevant details changed). Please indicate if something is wrong or mistaken.

Is there a public list of the other planned types of announcements? (prompted by https://meta.wikimedia.org/wiki/Talk:Tech/Server_switch_2018 ).

Probably not. But this is public, so let's list it here. In short, dealt with:

  • Included in Tech News (twice)
  • Posted on all Village Pumps (sometimes twice, first with an informal notice, later with the big message to everyone)
  • Prompted some of the big wikis that use them to use watchlist notices.
  • Set up a CentralNotice banner (as of writing, Monday: not set up yet, but had a video meeting with Seddon who is on the task)
  • Emailed mailing lists (wikitech-l, wmfall, GLAM and education mailing lists because they do outreach stuff – will email wikimedia-l too)
  • Added to the news on the Meta front page
  • Attempted to warn bot owners in a couple of places
  • Contacted the German chapter to let them do communications there

Pinged others about

  • IRC
  • Social media (Twitter, Facebook)

Decided to not do this time: a blog post.

Have also pinged Comms to remind them this is happening, and asked them if they wanted to give us a specific contact to reach out to should it for some reason prove necessary, otherwise we'll just use the general Comms email.

And those involved in the work around the European Parliament copyright vote, to make sure we don't cause problems for each other.