Page MenuHomePhabricator

Replicate babel db table on Labs
Closed, ResolvedPublic

Description

Replicate babel table on Labs, as it is apparently being not at the moment.

Details

Related Gerrit Patches:
operations/puppet : productionlabsdb: Add babel table to public views

Event Timeline

Base created this task.Mar 16 2017, 10:25 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 16 2017, 10:25 PM
jcrespo added a subscriber: jcrespo.

I've checked and babel table and it is being replicated to labs, just not exposed (needs view changes). I would suggest to labs team to ask for the ok from legal and/or security to expose it (it is not 100% clear to me if that is fully public data or not). Ping me back if they have concerns.

@Base - while it is of course not required -your request it clear enough-, requests like this are highly accelerated if a patch to https://phabricator.wikimedia.org/diffusion/OPUP/browse/production/modules/role/templates/labs/db/views/maintain-views.yaml is sent, in case you have the time.

bd808 added a subscriber: bd808.
> describe babel;
+-------------+------------------+------+-----+---------+-------+
| Field       | Type             | Null | Key | Default | Extra |
+-------------+------------------+------+-----+---------+-------+
| babel_user  | int(10) unsigned | NO   | PRI | NULL    |       |
| babel_lang  | varbinary(10)    | NO   | PRI | NULL    |       |
| babel_level | varbinary(2)     | NO   |     | NULL    |       |
+-------------+------------------+------+-----+---------+-------+

Exposing this is similar to exposing user preferences, but these rows are added via templates on user pages and are intended to help advertise multilingual communication abilities. Exposing these tables in the Labs replicas would allow creation/improvement of tools that help people figure out how to interact with multi-lingual communities. Adding WMF-Legal and Security-Team for what should be a quick sign off.

bd808 moved this task from Triage to Backlog on the Cloud-Services board.Mar 26 2017, 8:18 PM
bd808 moved this task from Backlog to Database on the Cloud-Services board.Mar 26 2017, 8:34 PM

All this is public info (available already via API action=query&meta=babel, or just user page). It is totally ok to make this public from a security perspective.

Approved by Legal. Thanks, all.

bd808 triaged this task as Normal priority.Jul 5 2017, 10:11 PM
bd808 removed projects: Security-Team, WMF-Legal.

Approved by Legal and Security.
The table needs to be added to maintain-views.yaml and then run in all the appropriate places.

Change 363492 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/puppet@production] labsdb: Add babel table to public views

https://gerrit.wikimedia.org/r/363492

Change 363492 merged by Andrew Bogott:
[operations/puppet@production] labsdb: Add babel table to public views

https://gerrit.wikimedia.org/r/363492

Andrew added a subscriber: Andrew.Jul 6 2017, 7:09 PM

This is done on most labsdbs. Remaining steps:

  1. Figure out why puppet is disabled on labsdb1009 and 1010
  2. Enable puppet, run puppet
  3. run "maintain-views --all-databases --table babel --debug" on those two hosts

Hi Andrew,

labsdb1009 and labsdb1010 have puppet disabled because:

The last Puppet run was at Tue Jul  4 12:17:48 UTC 2017 (3916 minutes ago). Puppet is disabled. myloader running

labsdb1009 myloader import process finished over night, so I have enabled (and run) puppet there.
I have also run maintain-views --all-databases --table babel --debug (you might want to double check if went fine, no errors were shown though)

labsdb1010 is still running myloader so puppet remains stopped.
Sorry for the troubles!

@Andrew in order not to block you on this task anymore, this is what I have done on labsdb1010 (which its import keeps running and will probably run for most of the day today still):

  • enabled puppet
  • run puppet
  • run maintain-views --all-databases --table babel --debug
  • disabled puppet
chasemp added a subscriber: chasemp.Jul 7 2017, 3:40 PM

Thanks @Marostegui you are a scholar and a gentlemen.

Is this task done now? @Base can you confirm?

bd808 assigned this task to Andrew.Jul 7 2017, 5:54 PM
bd808 moved this task from Important to Doing on the cloud-services-team (Kanban) board.
bd808 closed this task as Resolved.Jul 7 2017, 6:01 PM
bd808 moved this task from Doing to Done on the cloud-services-team (Kanban) board.

Checked with first alphabetic wiki from each shard:

$ sql enwiki
(u3518@enwiki.labsdb) [enwiki_p]> describe babel;
+-------------+------------------+------+-----+---------+-------+
| Field       | Type             | Null | Key | Default | Extra |
+-------------+------------------+------+-----+---------+-------+
| babel_user  | int(10) unsigned | NO   |     | NULL    |       |
| babel_lang  | varbinary(10)    | NO   |     | NULL    |       |
| babel_level | varbinary(2)     | NO   |     | NULL    |       |
+-------------+------------------+------+-----+---------+-------+
$ sql bgwiki
(u3518@bgwiki.labsdb) [bgwiki_p]> describe babel;
+-------------+------------------+------+-----+---------+-------+
| Field       | Type             | Null | Key | Default | Extra |
+-------------+------------------+------+-----+---------+-------+
| babel_user  | int(10) unsigned | NO   |     | NULL    |       |
| babel_lang  | varbinary(10)    | NO   |     | NULL    |       |
| babel_level | varbinary(2)     | NO   |     | NULL    |       |
+-------------+------------------+------+-----+---------+-------+
3 rows in set (0.00 sec)
$ sql aawiki
(u3518@aawiki.labsdb) [aawiki_p]> describe babel;
+-------------+------------------+------+-----+---------+-------+
| Field       | Type             | Null | Key | Default | Extra |
+-------------+------------------+------+-----+---------+-------+
| babel_user  | int(10) unsigned | NO   |     | NULL    |       |
| babel_lang  | varbinary(10)    | NO   |     | NULL    |       |
| babel_level | varbinary(2)     | NO   |     | NULL    |       |
+-------------+------------------+------+-----+---------+-------+
3 rows in set (0.00 sec)
$ sql commonswiki
(u3518@commonswiki.labsdb) [commonswiki_p]> describe babel;
+-------------+------------------+------+-----+---------+-------+
| Field       | Type             | Null | Key | Default | Extra |
+-------------+------------------+------+-----+---------+-------+
| babel_user  | int(10) unsigned | NO   |     | NULL    |       |
| babel_lang  | varbinary(10)    | NO   |     | NULL    |       |
| babel_level | varbinary(2)     | NO   |     | NULL    |       |
+-------------+------------------+------+-----+---------+-------+
3 rows in set (0.00 sec)
$ sql dewiki
(u3518@dewiki.labsdb) [dewiki_p]> describe babel;
+-------------+------------------+------+-----+---------+-------+
| Field       | Type             | Null | Key | Default | Extra |
+-------------+------------------+------+-----+---------+-------+
| babel_user  | int(10) unsigned | NO   |     | NULL    |       |
| babel_lang  | varbinary(10)    | NO   |     | NULL    |       |
| babel_level | varbinary(2)     | NO   |     | NULL    |       |
+-------------+------------------+------+-----+---------+-------+
3 rows in set (0.00 sec)
$ sql frwiki
(u3518@frwiki.labsdb) [frwiki_p]> describe babel;
+-------------+------------------+------+-----+---------+-------+
| Field       | Type             | Null | Key | Default | Extra |
+-------------+------------------+------+-----+---------+-------+
| babel_user  | int(10) unsigned | NO   |     | NULL    |       |
| babel_lang  | varbinary(10)    | NO   |     | NULL    |       |
| babel_level | varbinary(2)     | NO   |     | NULL    |       |
+-------------+------------------+------+-----+---------+-------+
3 rows in set (0.00 sec)
$ sql arwiki
(u3518@arwiki.labsdb) [arwiki_p]> describe babel;
+-------------+------------------+------+-----+---------+-------+
| Field       | Type             | Null | Key | Default | Extra |
+-------------+------------------+------+-----+---------+-------+
| babel_user  | int(10) unsigned | NO   |     | NULL    |       |
| babel_lang  | varbinary(10)    | NO   |     | NULL    |       |
| babel_level | varbinary(2)     | NO   |     | NULL    |       |
+-------------+------------------+------+-----+---------+-------+
3 rows in set (0.00 sec)