Help communicate read-only time for dewiki and wikidata for database split
Closed, ResolvedPublic

Subscribers
Tokens
"Like" token, awarded by Johan."Grey Medal" token, awarded by RandomDSdevel."The World Burns" token, awarded by Honischboy."Like" token, awarded by Liuxinyu970226."Like" token, awarded by Elitre.
Assigned To
Authored By
Marostegui, Nov 29 2017

Description

We are ready to split dewiki and wikidata into a separate set of servers (T177208).
In order to do that, we would need a 30 minutes period of read-only time on s5 (that means dewiki and wikidata) so we can do the split.

Date: 9th January 2018
Time: 06:00 UTC - 06:30 UTC

User impact: During this period writes will not be allowed. That means edits will not be possible on dewiki and on wikidata
Reads will continue as normal. They will not be impacted. So everyone will be able to browse those two wikis without any issues.

No problem assigning me. It's a great way to get someone's attention.

I'm of the opinion that @Lea_Lacroix_WMDE would be far more capable of handling discussions with these communities than I. :) She has far more familiarity and experience. I am available to help if need be.

aude added a subscriber: aude.Nov 30 2017, 1:42 AM
Lea_Lacroix_WMDE added a comment.EditedNov 30 2017, 1:45 PM

I can do it on Wikidata :) Though I'll be completely offline starting from January 6th, so I can send a message a few days before, but not taking care of answering to the possible questions on the D-day.

@CKoerner_WMF Just to make sure: will you take care about German Wikipedia?

@Lea_Lacroix_WMDE If I can reuse your message to remain consistent and avoid my terrible German, I would be happy to help with outreach to the German Wikipedia.

Which means, we should probably split this task into two separate tasks for each community. :)

@CKoerner_WMF No worries, I can do both, it's easier :)

Elitre awarded a token.Dec 7 2017, 2:33 PM

@JStrodt_WMDE I'll take care of Wikidata (first message on 05.01, ping on 09.01). Could you take care of German Wikipedia?

As a note, on some parts of the world, in particular, SFO, it will be still the 8th.

Change 401434 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Set s5 on read_only

https://gerrit.wikimedia.org/r/401434

Change 401436 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Point wikidatawiki to s8

https://gerrit.wikimedia.org/r/401436

Maybe it is too late now? This is happening in exactly 1 week.

I think it's not useful in that case. Messages on Wikidata and German Wikipedia will be posted this week.

That seems right, according to https://meta.wikimedia.org/wiki/Tech/News/For_contributors#What_is_typically_not_included : "Changes only relevant for one language or one wiki"

Except this is 2 wikis, and Wikidata is multilingual and with people coming from different communities. But that's just my2c :)

Maybe it is relevant, as the context why read only is needed is "to provide dedicated database resources to Wikidata due to its growth", which is kind of a big deal?

Johan added a subscriber: Johan.Jan 3 2018, 3:39 PM

Sometimes Commons and Wikidata are exceptions to the rule about one or a couple of wikis, as a lot of other wikis depend on them – I've clarified the documentation. For example, this will affect the ability to edit the language links on other wikis, yes?

Message sent on Wikidata, and to the main Wikidata mailing-lists. German Wikipedia will follow soon.

Johan added a comment.Jan 3 2018, 4:00 PM

I've added a short item in Tech/News/2018/02, please make sure it looks OK.

Just to clarify, will affect other wikis during the migration task?. I've read the announcement in Wikidata-L but I don't know if post a message in the Café (Village Pump) at es.wikipedia with the information.

Just to clarify, will affect other wikis during the migration task?. I've read the announcement in Wikidata-L but I don't know if post a message in the Café (Village Pump) at es.wikipedia with the information.

This will only affect dewiki and wikidata

Just to clarify, will affect other wikis during the migration task?. I've read the announcement in Wikidata-L but I don't know if post a message in the Café (Village Pump) at es.wikipedia with the information.

This will only affect dewiki and wikidata

Thanks!

this will affect the ability to edit the language links on other wikis, yes?

It will indeed affect that, as that is now entirely handled on Wikidata.

Doe read-only mode also mean that you can't change preferences & account passwords/mail-addresses and cannot add pages to your watchlist? Or is it just about editing pages?

Anything that would edit the database would either be disabled or generate an error- preferences and watchlist edition is handled on the database, so it would not be available. It is important to note that while we require a 30 minute window, we expect, if things go well, for the read only time to only take a couple of minutes (like previous times: T156008#2971613).

We schedule 30 minutes because we do not know exactly within those 30 minutes when it will start, and if things go wrong, it could be extended- we have not setup so many servers at the same time before, an we haven't ever moved a wiki so large before.

Honischboy removed a subscriber: Honischboy.
Silraks added a subscriber: Silraks.Jan 8 2018, 4:57 PM
revi added a subscriber: revi.Jan 8 2018, 7:22 PM
Izno added a subscriber: Izno.Jan 8 2018, 11:40 PM

What happens if I try to move a page on en.wp during the downtime? Will Wikidata update later?

So the job to update wikidata fails and probably gets retried soon. There is an slight chance that this will end up as complete failure. I should write some checks in the logs to make sure that gets fixed later.

@Izno in the event of an error happening, because the retry that Ladsgroup says also fails, and it is not fixed by us later, a purge or null edit on the origin wiki should fix any inconsistency. Sadly, this is theoretically impossible to fix unless we block as read only all wikis. In practice the problems should be very few.

BTW, this is starting in less than 20 minutes.

Change 401434 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Set s5 on read_only

https://gerrit.wikimedia.org/r/401434

Change 403104 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Set s5, s8 read only OFF

https://gerrit.wikimedia.org/r/403104

Mentioned in SAL (#wikimedia-operations) [2018-01-09T06:01:48Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Set s5 on read-only to start failover T177208 T181645 (duration: 00m 50s)

Change 401436 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Point wikidatawiki to s8

https://gerrit.wikimedia.org/r/401436

Mentioned in SAL (#wikimedia-operations) [2018-01-09T06:11:55Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Splitting s5 and s8 T177208 T181645 (duration: 00m 50s)

Change 403104 merged by Marostegui:
[operations/mediawiki-config@master] db-eqiad.php: Set s5, s8 read only OFF

https://gerrit.wikimedia.org/r/403104

Mentioned in SAL (#wikimedia-operations) [2018-01-09T06:14:46Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove read_only from s5 and s8 T177208 T181645 (duration: 00m 27s)

Marostegui closed this task as Resolved.Jan 9 2018, 6:30 AM
Marostegui added a subscriber: Joe.

This has been done.
Read only started: 06:01
Read only finished: 06:14

Thanks @Ladsgroup @aude @mark @Joe for being online and supporting the DBAs!
Thanks everyone for handling this announcement

I believe this task can be closed

Further comments/issues should go to the parent task T177208

Johan awarded a token.Jan 9 2018, 6:47 AM