Page MenuHomePhabricator
Create Task
Maniphest T166205

Convert unique keys into primary keys for some wiki tables on s2
Closed, ResolvedPublic
Actions

Description

Pending hosts:

codfw - entire DC

To be run on codfw master and let it replicate db2017:

./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=categorylinks "add primary key (cl_from,cl_to)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=categorylinks "drop key cl_from" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=templatelinks "add primary key (tl_from,tl_namespace,tl_title)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=pagelinks "add primary key (pl_from,pl_namespace,pl_title)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=text "drop key old_id, add primary key (old_id)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=imagelinks "drop key il_from, add primary key (il_from,il_to)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=iwlinks "drop key iwl_from, add primary key (iwl_from,iwl_prefix,iwl_title)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=langlinks "drop key ll_from, add primary key (ll_from,ll_lang)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=log_search "drop key ls_field_val, add primary key (ls_field,ls_value,ls_log_id)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=math "drop key math_inputhash, add primary key (math_inputhash)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=module_deps "drop key md_module_skin, add primary key (md_module,md_skin)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=objectcache "drop key keyname, add primary key (keyname)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=querycache_info "drop key  qci_type, add primary key (qci_type)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=site_stats "drop key ss_row_id, add primary key (ss_row_id)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=transcache "drop key tc_url_idx, add primary key (tc_url)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=user_former_groups "drop key ufg_user_group, add primary key (ufg_user,ufg_group)" --method=ddl --replicate
./software/dbtools/osc_host.sh --host=db2017.codfw.wmnet --dblist=/home/marostegui/git/mediawiki-config/dblists/s2.dblist --table=user_properties "drop key user_properties_user_property, add primary key (up_user,up_property)" --method=ddl --replicate

eqiad: all hosts except db1054 (master) and db1090

./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=categorylinks "add primary key (cl_from,cl_to)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=categorylinks "drop key cl_from" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=templatelinks "add primary key (tl_from,tl_namespace,tl_title)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=pagelinks "add primary key (pl_from,pl_namespace,pl_title)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=text "drop key old_id, add primary key (old_id)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=imagelinks "drop key il_from, add primary key (il_from,il_to)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=iwlinks "drop key iwl_from, add primary key (iwl_from,iwl_prefix,iwl_title)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=langlinks "drop key ll_from, add primary key (ll_from,ll_lang)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=log_search "drop key ls_field_val, add primary key (ls_field,ls_value,ls_log_id)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=math "drop key math_inputhash, add primary key (math_inputhash)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=module_deps "drop key md_module_skin, add primary key (md_module,md_skin)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=objectcache "drop key keyname, add primary key (keyname)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=querycache_info "drop key  qci_type, add primary key (qci_type)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=site_stats "drop key ss_row_id, add primary key (ss_row_id)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=transcache "drop key tc_url_idx, add primary key (tc_url)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=user_former_groups "drop key ufg_user_group, add primary key (ufg_user,ufg_group)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=user_properties "drop key user_properties_user_property, add primary key (up_user,up_property)" --method=ddl --no-replicate

Hosts:

  • labsdb1001.eqiad.wmnet
  • labsdb1003.eqiad.wmnet
  • db1069.eqiad.wmnet - (Will not be done as s2 will go away from db1069 soon: T166546)
  • dbstore1002.eqiad.wmnet
  • dbstore1001.eqiad.wmnet
  • db1021.eqiad.wmnet
  • db1036.eqiad.wmnet
  • db1047.eqiad.wmnet (I will not alter db1047 because it will take many many days and this is going away at some point: T156844)
  • db1060.eqiad.wmnet
  • db1074.eqiad.wmnet
  • db1076.eqiad.wmnet
  • db1090.eqiad.wmnet
  • db1018.eqiad.wmnet
  • db1054.eqiad.wmnet

Details

SubjectRepoBranchLines +/-
db-eqiad.php: Depool db1021operations/mediawiki-configmaster+4 -4
db-eqiad.php: Add comment to db1018 statusoperations/mediawiki-configmaster+1 -1
db-eqiad.php: Depool db1036operations/mediawiki-configmaster+7 -7
db-eqiad.php: Depool db1060operations/mediawiki-configmaster+3 -3
db-eqiad.php: Depool db1074operations/mediawiki-configmaster+3 -3
db-eqiad.php: Depool db1076operations/mediawiki-configmaster+1 -1
Customize query in gerrit

Related Objects
Search...

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.
StatusSubtypeAssignedTask
· · ·
ResolvedMarosteguiT164185 Convert unique keys into primary keys for some wiki tables on s1, s2, s4, s5 and s7 (eqiad)
ResolvedMarosteguiT166205 Convert unique keys into primary keys for some wiki tables on s2
· · ·

Event Timeline

Marostegui created this task.May 24 2017, 8:29 AM
Marostegui moved this task from Triage to Pending comment on the DBA board.May 24 2017, 8:35 AM
Marostegui removed a project: Schema-change-in-production.Jun 2 2017, 10:12 AM
Marostegui updated the task description. (Show Details)
Marostegui moved this task from Pending comment to In progress on the DBA board.Jun 5 2017, 7:14 AM
Stashbot added a comment.Jun 5 2017, 7:15 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-05T07:15:12Z] <marostegui> Deploy alter table in s2 (codfw master) this will generate lag in codfw - T166205

Marostegui added a comment.Jun 7 2017, 5:53 AM

codfw master is done and replicated downstream

gerritbot subscribed.Jun 7 2017, 8:54 AM

Change 357566 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1076

https://gerrit.wikimedia.org/r/357566

gerritbot added a project: Patch-For-Review.Jun 7 2017, 8:54 AM
gerritbot added a comment.Jun 7 2017, 8:57 AM

Change 357566 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1076

https://gerrit.wikimedia.org/r/357566

Stashbot added a comment.Jun 7 2017, 8:58 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-07T08:58:34Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1076 - T166205 (duration: 00m 43s)

Stashbot added a comment.Jun 7 2017, 8:58 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-07T08:58:52Z] <marostegui> Deploy alter table on s2 - db1076 - T166205

Marostegui updated the task description. (Show Details)Jun 8 2017, 5:39 AM
Stashbot added a comment.Jun 8 2017, 5:43 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-08T05:43:39Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1076 - T166205 (duration: 00m 45s)

gerritbot added a comment.Jun 8 2017, 5:49 AM

Change 357757 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1074

https://gerrit.wikimedia.org/r/357757

gerritbot added a comment.Jun 8 2017, 5:52 AM

Change 357757 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1074

https://gerrit.wikimedia.org/r/357757

Stashbot added a comment.Jun 8 2017, 5:53 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-08T05:52:59Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1074 - T166205 (duration: 00m 43s)

Stashbot added a comment.Jun 8 2017, 5:54 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-08T05:54:24Z] <marostegui> Deploy alter table s2 - db1074 - T166205

Marostegui updated the task description. (Show Details)Jun 9 2017, 5:46 AM
Stashbot added a comment.Jun 9 2017, 5:51 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-09T05:51:08Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1074 - T166205 (duration: 00m 42s)

gerritbot added a comment.Jun 12 2017, 7:26 AM

Change 358315 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1060

https://gerrit.wikimedia.org/r/358315

gerritbot added a comment.Jun 12 2017, 7:29 AM

Change 358315 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1060

https://gerrit.wikimedia.org/r/358315

Stashbot added a comment.Jun 12 2017, 7:31 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-12T07:31:09Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1060 - T166205 (duration: 00m 41s)

Stashbot added a comment.Jun 12 2017, 7:31 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-12T07:31:57Z] <marostegui> Deploy alter table s2 - db1060 - T166205

Marostegui updated the task description. (Show Details)Jun 13 2017, 8:34 AM
Stashbot added a comment.Jun 13 2017, 8:42 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-13T08:42:29Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1060 - T166205 (duration: 00m 41s)

gerritbot added a comment.Jun 13 2017, 8:48 AM

Change 358529 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1036

https://gerrit.wikimedia.org/r/358529

Stashbot added a comment.Jun 13 2017, 12:12 PM

Mentioned in SAL (#wikimedia-operations) [2017-06-13T12:12:51Z] <marostegui> Deploy alter table on s2 on dbstore1002 - T166205

gerritbot added a comment.Jun 13 2017, 12:38 PM

Change 358529 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1036

https://gerrit.wikimedia.org/r/358529

Stashbot added a comment.Jun 13 2017, 12:39 PM

Mentioned in SAL (#wikimedia-operations) [2017-06-13T12:39:11Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1036 - T166205 (duration: 00m 41s)

Stashbot added a comment.Jun 13 2017, 12:44 PM

Mentioned in SAL (#wikimedia-operations) [2017-06-13T12:44:30Z] <marostegui> Deploy alter table on s2 on db1036 - T166205

Krinkle mentioned this in T167885: Regression: Backend Save Timing raised by about 50%.Jun 14 2017, 2:09 PM
Stashbot added a comment.Jun 15 2017, 5:22 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-15T05:22:49Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1036 - T166205 (duration: 00m 44s)

Marostegui updated the task description. (Show Details)Jun 15 2017, 5:23 AM
gerritbot added a comment.Jun 15 2017, 5:45 AM

Change 359092 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Add comment to db1018 status

https://gerrit.wikimedia.org/r/359092

gerritbot added a comment.Jun 15 2017, 5:48 AM

Change 359092 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Add comment to db1018 status

https://gerrit.wikimedia.org/r/359092

Stashbot added a comment.Jun 15 2017, 5:49 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-15T05:49:53Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Add comments to db1018 current status - T166205 (duration: 00m 43s)

Stashbot added a comment.Jun 15 2017, 5:50 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-15T05:50:40Z] <marostegui> Deploy alter table s2 - db1018 - T166205

Stashbot added a comment.Jun 15 2017, 6:08 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-15T06:08:06Z] <marostegui> Deploy alter table s2 - labsdb1003 - T166205

Marostegui added a comment.Jun 15 2017, 6:08 AM

cloud-services-team please note that this alter has been started on labsdb1003 so delays are expected on the s2 instance for a couple of days or so.

Marostegui updated the task description. (Show Details)Jun 15 2017, 8:19 AM
Marostegui updated the task description. (Show Details)Jun 16 2017, 1:30 PM
Stashbot added a comment.Jun 16 2017, 1:36 PM

Mentioned in SAL (#wikimedia-operations) [2017-06-16T13:36:33Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Remove comments from db1018 current status - T166205 (duration: 00m 41s)

Marostegui updated the task description. (Show Details)Jun 16 2017, 2:32 PM
gerritbot added a comment.Jun 19 2017, 5:58 AM

Change 359901 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1021

https://gerrit.wikimedia.org/r/359901

Marostegui updated the task description. (Show Details)Jun 19 2017, 6:00 AM
gerritbot added a comment.Jun 19 2017, 6:16 AM

Change 359901 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1021

https://gerrit.wikimedia.org/r/359901

Stashbot added a comment.Jun 19 2017, 6:18 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-19T06:18:46Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1021 - T166205 (duration: 00m 41s)

Stashbot added a comment.Jun 19 2017, 6:23 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-19T06:23:08Z] <marostegui> Deploy alter table on s2 - db1021 - T166205

Marostegui added a comment.Jun 19 2017, 6:37 AM

cloud-services-team I have started to alter labsdb1001 for the s2 shard - this might result on labsdb1001 being delayed for a couple of days.

Stashbot added a comment.Jun 19 2017, 6:38 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-19T06:38:06Z] <marostegui> Deploy alter table s2 - labsdb1001 - T166205

Stashbot added a comment.Jun 19 2017, 9:30 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-19T09:30:33Z] <marostegui> Deploy alter table on s2 - dbstore1001 - T166205

Marostegui updated the task description. (Show Details)Jun 21 2017, 5:45 AM
Stashbot added a comment.Jun 21 2017, 5:50 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-21T05:50:34Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1021 - T166205 (duration: 01m 00s)

Marostegui closed this task as Resolved.Jun 21 2017, 12:55 PM
Marostegui updated the task description. (Show Details)

Everything is now done.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL