Page MenuHomePhabricator

Convert unique keys into primary keys for some wiki tables on s2
Closed, ResolvedPublic

Description

Pending hosts:

codfw - entire DC

To be run on codfw master and let it replicate db2017:

./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=categorylinks "add primary key (cl_from,cl_to)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=categorylinks "drop key cl_from" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=templatelinks "add primary key (tl_from,tl_namespace,tl_title)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=pagelinks "add primary key (pl_from,pl_namespace,pl_title)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=text "drop key old_id, add primary key (old_id)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=imagelinks "drop key il_from, add primary key (il_from,il_to)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=iwlinks "drop key iwl_from, add primary key (iwl_from,iwl_prefix,iwl_title)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=langlinks "drop key ll_from, add primary key (ll_from,ll_lang)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=log_search "drop key ls_field_val, add primary key (ls_field,ls_value,ls_log_id)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=math "drop key math_inputhash, add primary key (math_inputhash)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=module_deps "drop key md_module_skin, add primary key (md_module,md_skin)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=objectcache "drop key keyname, add primary key (keyname)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=querycache_info "drop key  qci_type, add primary key (qci_type)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=site_stats "drop key ss_row_id, add primary key (ss_row_id)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=transcache "drop key tc_url_idx, add primary key (tc_url)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=user_former_groups "drop key ufg_user_group, add primary key (ufg_user,ufg_group)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=user_properties "drop key user_properties_user_property, add primary key (up_user,up_property)" --method=ddl --replicate

eqiad: all hosts except db1054 (master) and db1090

./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=categorylinks "add primary key (cl_from,cl_to)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=categorylinks "drop key cl_from" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=templatelinks "add primary key (tl_from,tl_namespace,tl_title)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=pagelinks "add primary key (pl_from,pl_namespace,pl_title)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=text "drop key old_id, add primary key (old_id)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=imagelinks "drop key il_from, add primary key (il_from,il_to)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=iwlinks "drop key iwl_from, add primary key (iwl_from,iwl_prefix,iwl_title)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=langlinks "drop key ll_from, add primary key (ll_from,ll_lang)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=log_search "drop key ls_field_val, add primary key (ls_field,ls_value,ls_log_id)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=math "drop key math_inputhash, add primary key (math_inputhash)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=module_deps "drop key md_module_skin, add primary key (md_module,md_skin)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=objectcache "drop key keyname, add primary key (keyname)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=querycache_info "drop key  qci_type, add primary key (qci_type)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=site_stats "drop key ss_row_id, add primary key (ss_row_id)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=transcache "drop key tc_url_idx, add primary key (tc_url)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=user_former_groups "drop key ufg_user_group, add primary key (ufg_user,ufg_group)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=user_properties "drop key user_properties_user_property, add primary key (up_user,up_property)" --method=ddl --no-replicate

Hosts:

  • labsdb1001.eqiad.wmnet
  • labsdb1003.eqiad.wmnet
  • db1069.eqiad.wmnet - (Will not be done as s2 will go away from db1069 soon: T166546)
  • dbstore1002.eqiad.wmnet
  • dbstore1001.eqiad.wmnet
  • db1021.eqiad.wmnet
  • db1036.eqiad.wmnet
  • db1047.eqiad.wmnet (I will not alter db1047 because it will take many many days and this is going away at some point: T156844)
  • db1060.eqiad.wmnet
  • db1074.eqiad.wmnet
  • db1076.eqiad.wmnet
  • db1090.eqiad.wmnet
  • db1018.eqiad.wmnet
  • db1054.eqiad.wmnet

Details

Related Gerrit Patches:
operations/mediawiki-config : masterdb-eqiad.php: Depool db1021
operations/mediawiki-config : masterdb-eqiad.php: Add comment to db1018 status
operations/mediawiki-config : masterdb-eqiad.php: Depool db1036
operations/mediawiki-config : masterdb-eqiad.php: Depool db1060
operations/mediawiki-config : masterdb-eqiad.php: Depool db1074
operations/mediawiki-config : masterdb-eqiad.php: Depool db1076

Related Objects

Event Timeline

Marostegui moved this task from Triage to Next on the DBA board.May 24 2017, 8:35 AM
Marostegui updated the task description. (Show Details)
Marostegui moved this task from Next to In progress on the DBA board.Jun 5 2017, 7:14 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-05T07:15:12Z] <marostegui> Deploy alter table in s2 (codfw master) this will generate lag in codfw - T166205

codfw master is done and replicated downstream

Change 357566 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1076

https://gerrit.wikimedia.org/r/357566

Change 357566 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1076

https://gerrit.wikimedia.org/r/357566

Mentioned in SAL (#wikimedia-operations) [2017-06-07T08:58:34Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1076 - T166205 (duration: 00m 43s)

Mentioned in SAL (#wikimedia-operations) [2017-06-07T08:58:52Z] <marostegui> Deploy alter table on s2 - db1076 - T166205

Marostegui updated the task description. (Show Details)Jun 8 2017, 5:39 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-08T05:43:39Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1076 - T166205 (duration: 00m 45s)

Change 357757 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1074

https://gerrit.wikimedia.org/r/357757

Change 357757 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1074

https://gerrit.wikimedia.org/r/357757

Mentioned in SAL (#wikimedia-operations) [2017-06-08T05:52:59Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1074 - T166205 (duration: 00m 43s)

Mentioned in SAL (#wikimedia-operations) [2017-06-08T05:54:24Z] <marostegui> Deploy alter table s2 - db1074 - T166205

Marostegui updated the task description. (Show Details)Jun 9 2017, 5:46 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-09T05:51:08Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1074 - T166205 (duration: 00m 42s)

Change 358315 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1060

https://gerrit.wikimedia.org/r/358315

Change 358315 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1060

https://gerrit.wikimedia.org/r/358315

Mentioned in SAL (#wikimedia-operations) [2017-06-12T07:31:09Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1060 - T166205 (duration: 00m 41s)

Mentioned in SAL (#wikimedia-operations) [2017-06-12T07:31:57Z] <marostegui> Deploy alter table s2 - db1060 - T166205

Marostegui updated the task description. (Show Details)Jun 13 2017, 8:34 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-13T08:42:29Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1060 - T166205 (duration: 00m 41s)

Change 358529 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1036

https://gerrit.wikimedia.org/r/358529

Mentioned in SAL (#wikimedia-operations) [2017-06-13T12:12:51Z] <marostegui> Deploy alter table on s2 on dbstore1002 - T166205

Change 358529 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1036

https://gerrit.wikimedia.org/r/358529

Mentioned in SAL (#wikimedia-operations) [2017-06-13T12:39:11Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1036 - T166205 (duration: 00m 41s)

Mentioned in SAL (#wikimedia-operations) [2017-06-13T12:44:30Z] <marostegui> Deploy alter table on s2 on db1036 - T166205

Mentioned in SAL (#wikimedia-operations) [2017-06-15T05:22:49Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1036 - T166205 (duration: 00m 44s)

Marostegui updated the task description. (Show Details)Jun 15 2017, 5:23 AM

Change 359092 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Add comment to db1018 status

https://gerrit.wikimedia.org/r/359092

Change 359092 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Add comment to db1018 status

https://gerrit.wikimedia.org/r/359092

Mentioned in SAL (#wikimedia-operations) [2017-06-15T05:49:53Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Add comments to db1018 current status - T166205 (duration: 00m 43s)

Mentioned in SAL (#wikimedia-operations) [2017-06-15T05:50:40Z] <marostegui> Deploy alter table s2 - db1018 - T166205

Mentioned in SAL (#wikimedia-operations) [2017-06-15T06:08:06Z] <marostegui> Deploy alter table s2 - labsdb1003 - T166205

cloud-services-team please note that this alter has been started on labsdb1003 so delays are expected on the s2 instance for a couple of days or so.

Marostegui updated the task description. (Show Details)Jun 15 2017, 8:19 AM
Marostegui updated the task description. (Show Details)Jun 16 2017, 1:30 PM

Mentioned in SAL (#wikimedia-operations) [2017-06-16T13:36:33Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove comments from db1018 current status - T166205 (duration: 00m 41s)

Marostegui updated the task description. (Show Details)Jun 16 2017, 2:32 PM

Change 359901 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1021

https://gerrit.wikimedia.org/r/359901

Marostegui updated the task description. (Show Details)Jun 19 2017, 6:00 AM

Change 359901 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1021

https://gerrit.wikimedia.org/r/359901

Mentioned in SAL (#wikimedia-operations) [2017-06-19T06:18:46Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1021 - T166205 (duration: 00m 41s)

Mentioned in SAL (#wikimedia-operations) [2017-06-19T06:23:08Z] <marostegui> Deploy alter table on s2 - db1021 - T166205

cloud-services-team I have started to alter labsdb1001 for the s2 shard - this might result on labsdb1001 being delayed for a couple of days.

Mentioned in SAL (#wikimedia-operations) [2017-06-19T06:38:06Z] <marostegui> Deploy alter table s2 - labsdb1001 - T166205

Mentioned in SAL (#wikimedia-operations) [2017-06-19T09:30:33Z] <marostegui> Deploy alter table on s2 - dbstore1001 - T166205

Marostegui updated the task description. (Show Details)Jun 21 2017, 5:45 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-21T05:50:34Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1021 - T166205 (duration: 01m 00s)

Marostegui closed this task as Resolved.Jun 21 2017, 12:55 PM
Marostegui updated the task description. (Show Details)

Everything is now done.