Page MenuHomePhabricator

Prepare and check storage layer for id_internalwikimedia
Closed, ResolvedPublic

Description

The new wiki is going to be private.

Event Timeline

Urbanecm created this task.Jun 8 2018, 3:10 PM
Restricted Application added a project: User-Urbanecm. · View Herald TranscriptJun 8 2018, 3:10 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Let us know when this is created, we need to create the filters for it, restart sanitariums and sanitize it

This will cause a deadlock :). This is a private wiki and private wikis cannot be created without green light from DBAs.

Urbanecm removed Urbanecm as the assignee of this task.Jun 8 2018, 3:12 PM

Eh, assigned to myself by mistake.

This will cause a deadlock :). This is a private wiki and private wikis cannot be created without green light from DBAs.

Yeah, I will push the patch for the filters, restart sanitariums and all that. So I will assign it to myself to block it on me

Change 438264 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] realm.pp: Add idprivatewikimedia as private wiki

https://gerrit.wikimedia.org/r/438264

Change 438264 merged by Marostegui:
[operations/puppet@production] realm.pp: Add idprivatewikimedia as private wiki

https://gerrit.wikimedia.org/r/438264

Mentioned in SAL (#wikimedia-operations) [2018-06-08T15:26:36Z] <marostegui> Restart MySQL on codfw sanitarium hosts db2094, db2095 - https://phabricator.wikimedia.org/T196748

Marostegui triaged this task as Normal priority.

I have merged the patch and restarted MySQL on codfw sanitariums, the filter has been applied there.
On Monday I will restart eqiad ones (which are the actives one).

This comment was removed by Marostegui.

Ok, thanks!

jcrespo added a subscriber: jcrespo.Jun 8 2018, 4:03 PM

So, I think everybody knows what is the process like and it is documented, but just to clarify in case someone else reads this ticket:

  • If a new private wiki is going to be setup- deployers must be blocked on us to deploy replication filters and not continue. After that, ping us after creation to double check again the filters work as intended
  • If a new non-private wiki is going to be setup, just give us a heads up and continue, until the creation actually happens, when you need to give us a heads up so those tables can be exposed.

Note there is enough safeguards to not leak private data if the process is not followed by mistake, but better be sure in the first place. With time, we expect to be much more transparent in the future.

Urbanecm moved this task from Backlog to Watching on the User-Urbanecm board.Jun 8 2018, 4:14 PM
Urbanecm renamed this task from Prepare and check storage layer for idprivatewikimedia to Prepare and check storage layer for id_privatewikimedia.Jun 8 2018, 5:53 PM

@Marostegui Sorry, but I've made a mistake in the DB name. It should be id_privatewikimedia, not idprivatewikimedia. Can you please reapply the filters?

Thanks for the heads up. I will do it on Monday

Thank you and sorry again.

We don't tend to like databases with _ because that is a special character, and while it is complete legal and allowed, sometimes it causes confusion on poorly written scripts. Are you sure you want a database with an underscore?

@jcrespo I'm pretty sure, arbcom-cs.wikipedia.org is named arbcom_cswiki, noboard-chapters.wikimedia.org is named noboard_chapterswikimedia, be-x-old.wikipedia.org is named be_x_oldwiki, pa-us.wikimedia.org is named pa_uswikimedia, wg-en.wikipedia.org is named wg_enwiki and so on. There seems to be a rule that if there's a dash in domain name, there's an underscore in database name in the same place. But maybe this is a former standard that is being changed in some task, don't know.

Urbanecm added a comment.EditedJun 8 2018, 6:01 PM

If there are scripts that have problems with legal characters, the scripts should be rewritten IMHO. For example, I really cannot expect not using "a" in db name, because some (very) poorly written scripts cannot work with "a" in a dbname (of course, I used argumentum ad absurdum in this comment).

@Urbanecm I don't disagree, but sadly, those are related to scripts run by wikireplica users, so nothing we can control, but later come to us to complain :-(

See you!

Oh, ok, understood. I thought you are reffering to maintenance scripts in ME. BTW, this is a private wiki and having it accessible from toolforge would be a big problem anyway.

Urbanecm renamed this task from Prepare and check storage layer for id_privatewikimedia to Prepare and check storage layer for id_internalwikimedia.Jun 10 2018, 7:44 AM

Finally, they decided to use id-internal.wikimedia.org, so @Marostegui, when amending the filters, please use this new dbname. Sorry for second changing!

Marostegui added a comment.EditedJun 10 2018, 9:14 AM

Thanks for the heads up. I will create the filter for id_internalwikimedia and restart all sanitariums on Monday.

Change 439524 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] realm.pp: Add id_internalwikimedia as private wiki

https://gerrit.wikimedia.org/r/439524

Change 439524 merged by Marostegui:
[operations/puppet@production] realm.pp: Add id_internalwikimedia as private wiki

https://gerrit.wikimedia.org/r/439524

Mentioned in SAL (#wikimedia-operations) [2018-06-11T05:25:12Z] <marostegui> Restart mysql on codfw sanitariums (db2094, db2095) to pick up new replication filters - T196748

Mentioned in SAL (#wikimedia-operations) [2018-06-11T05:32:34Z] <marostegui> Restart mysql on codfw sanitariums (db1095, db1102, db1124, db1125) to pick up new replication filters - T196748

Marostegui removed Marostegui as the assignee of this task.Jun 11 2018, 5:49 AM
Marostegui moved this task from In progress to Done on the DBA board.

All sanitariums host have been restarted and they've picked up the filters. This is now fine from the DBA side.
Please, do ping us once the wiki has been created to make sure we double check it is all being redacted finely.

Marostegui added a subscriber: Reedy.

Removing cloud-services as there is nothing for them to do here as it is private.

Vvjjkkii renamed this task from Prepare and check storage layer for id_internalwikimedia to 3dbaaaaaaa.Jul 1 2018, 1:05 AM
Vvjjkkii raised the priority of this task from Normal to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed subscribers: gerritbot, Aklapper.
Marostegui renamed this task from 3dbaaaaaaa to Prepare and check storage layer for id_internalwikimedia.Jul 2 2018, 5:14 AM
Marostegui lowered the priority of this task from High to Normal.
Marostegui updated the task description. (Show Details)
Marostegui closed this task as Resolved.Aug 2 2018, 12:58 PM
Marostegui claimed this task.

This wiki was created in production:

root@db1075.eqiad.wmnet[id_internalwikimedia]> show databases like 'id_internal%';
+-------------------------+
| Database (id_internal%) |
+-------------------------+
| id_internalwikimedia    |
+-------------------------+
1 row in set (0.00 sec)

But the filters went fine and it is not on sanitariums:

mysql:root@localhost [(none)]> select @@hostname; show databases like 'id_internal%';
+------------+
| @@hostname |
+------------+
| db1124     |
+------------+
1 row in set (0.00 sec)

Empty set (0.00 sec)

mysql:root@localhost [(none)]> select @@hostname; show databases like 'id_internal%';
+------------+
| @@hostname |
+------------+
| db2094     |
+------------+
1 row in set (0.00 sec)

Empty set (0.00 sec)