Page MenuHomePhabricator

Review and create CX Parallel corpora table
Closed, ResolvedPublic

Description

Request for DBA review of https://gerrit.wikimedia.org/r/#/c/256385/ and creating the table for CX Parallel corpora.

Event Timeline

santhosh created this task.Dec 8 2015, 11:25 AM
santhosh assigned this task to jcrespo.
santhosh raised the priority of this task from to Normal.
santhosh updated the task description. (Show Details)
jcrespo moved this task from Triage to In progress on the DBA board.
santhosh moved this task from Backlog to In Review on the LE-CX7-Sprint 4 board.Dec 10 2015, 11:50 AM

@jcrespo We have addressed your comment about the schema(added primary key). We would like to get the table created by next tuesday(next train schedule).

To avoid issues to user we are turning it off this feature using a configuration and plans to enable as soon as table is created.

Amire80 moved this task from Needs Triage to CX7 on the ContentTranslation board.Dec 14 2015, 9:57 AM
santhosh moved this task from In Review to Blocked on the LE-CX7-Sprint 4 board.Dec 18 2015, 3:52 AM
santhosh set Security to None.
santhosh moved this task from Backlog to Blocked on the LE-CX7-Sprint break board.Dec 30 2015, 6:22 AM

@jcrespo, We are waiting for this.

jcrespo removed jcrespo as the assignee of this task.Jan 19 2016, 6:27 PM
jcrespo added a subscriber: jcrespo.

I cannot review a change that has been already merged.

jcrespo removed a project: DBA.Jan 19 2016, 6:28 PM
santhosh added a subscriber: DBA.Jan 20 2016, 4:10 AM

@jcrespo, you had reviewed the patch in December and we addressed your comments. What is pending is creating the tables.

You do not need me to create new tables, that is not a schema change and is not dangerous. See: https://wikitech.wikimedia.org/wiki/Schema_changes#What_is_not_a_schema_change I had already agreed with the table structure.

@jcrespo, Thanks for updates.

Few questions:

  • Who can create tables? Anyone from Ops team can help?
  • Is there any documentation on creating tables on Wikitech?

Let me try to get you help from other developers/devops.

hashar added a subscriber: hashar.

Jaime has agreed on the table schema and adding a new table is not a schema change. Ie it is not being handled by DBA since it is low risk. Though the wiki page states a table creation on all DB should be notified to dba.

Then how do we create a new table? Almost as you would do on a normal wiki / developer instance. Use MediaWiki maintenance/sql.php to source the .sql patch file that deals with the table creation. The process is documented on wikitech - how to do a schema change:

mwscript sql.php --wiki=testwiki extensions/ContentTranslation/sql/parallel-corpora.sql

Might be tested on the Beta-Cluster-Infrastructure , though there update.php is run continuously. So to do the test you would have to drop the table first or run the sql.php against a wiki database that does not have the table.

Do not ever run update.php on production wiki.

Feel free to poke Release-Engineering-Team about it on IRC #wikimedia-releng or poke regular SWAT deployers which should be familiar with sql.php usage.

(to clarify: I am not going to handle the table creation but we can offer support / training :D )

The table belongs to the shared wikishared or extension1 (IIRC they have different names in production and beta labs) database, not to the database of any particular wiki. As far as I can see sql.php does not support this.

Maybe I can modify sql.php to allow this on a separate issue? It works for wiki creation, so I think it should be included here, too.

Indeed MediaWiki maintenance/sql.php relies on a $wgDBName. So instead you can use the mysql client wrapper which connect you directly to any arbitrary database name as an admin. The wrapper grab the user/pass by asking it from MediaWiki PrivateSettings.php

tin$ which sql
/usr/local/bin/sql
tin$ sql wikishared
mysql:wgDBUser@10.0.0.y [wikishared]>

That sql commands is the raw mysql client. It will not expand the MediaWiki magic comments such as /* _ */ or /*$wgDBTableOptions*/. So you will want to double check them and might have to create a suitable .sql file.

Then it should be:

tin$ sql wikishared
mysql:wgDBUser@10.0.0.y [wikishared]> source whatever.sql
// output of table creation

mwscript sql.php testwiki --cluster extension1 WFM.

jcrespo added a comment.EditedJan 20 2016, 4:03 PM

Oh, so no need to change it. @santhosh, follow Krenair's advice. Let me know if you need help (I knew I didn't know much about mediawiki's scripts, that why I requested help).

KartikMistry closed this task as Resolved.Jan 22 2016, 4:26 AM
KartikMistry moved this task from Blocked to Done on the LE-CX8-Sprint 1 board.