Page MenuHomePhabricator

Convert unique keys into primary keys for some wiki tables on s5
Closed, ResolvedPublic

Description

Pending hosts:

codfw - entire DC
eqiad: all hosts except db1063 (master) and db1092

./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=categorylinks "add primary key (cl_from,cl_to)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=categorylinks "drop key cl_from" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=templatelinks "add primary key (tl_from,tl_namespace,tl_title)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=pagelinks "add primary key (pl_from,pl_namespace,pl_title)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=text "drop key old_id, add primary key (old_id)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=imagelinks "drop key il_from, add primary key (il_from,il_to)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=iwlinks "drop key iwl_from, add primary key (iwl_from,iwl_prefix,iwl_title)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=langlinks "drop key ll_from, add primary key (ll_from,ll_lang)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=log_search "drop key ls_field_val, add primary key (ls_field,ls_value,ls_log_id)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=math "drop key math_inputhash, add primary key (math_inputhash)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=module_deps "drop key md_module_skin, add primary key (md_module,md_skin)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=objectcache "drop key keyname, add primary key (keyname)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=querycache_info "drop key  qci_type, add primary key (qci_type)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=site_stats "drop key ss_row_id, add primary key (ss_row_id)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=transcache "drop key tc_url_idx, add primary key (tc_url)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=user_former_groups "drop key ufg_user_group, add primary key (ufg_user,ufg_group)" --method=ddl --no-replicate
./software/dbtools/osc_host.sh --host=$host.eqiad.wmnet --dblist=$shard.dblist --table=user_properties "drop key user_properties_user_property, add primary key (up_user,up_property)" --method=ddl --no-replicate
  • labsdb1001
  • labsdb1003
  • db1069 (Will not be done as this host is going away soon)
  • labsdb1009
  • labsdb1010
  • labsdb1011
  • db1095
  • dbstore1001
  • dbstore1002
  • db1026
  • db1045
  • db1049
  • db1070
  • db1071
  • db1082
  • db1087
  • db1092
  • db1063

Details

Related Gerrit Patches:
operations/mediawiki-config : masterdb-eqiad.php: Depool db1026
operations/mediawiki-config : masterdb-eqiad.php: Depool db1045
operations/mediawiki-config : masterdb-eqiad.php: Depool db1070
operations/mediawiki-config : masterdb-eqiad.php: Depool db1082
operations/mediawiki-config : masterdb-eqiad.php: Depool db1087
operations/mediawiki-config : masterdb-eqiad.php: Depool db1071

Related Objects

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.

Event Timeline

Marostegui moved this task from Triage to Next on the DBA board.May 24 2017, 8:35 AM
Marostegui moved this task from Next to In progress on the DBA board.Jun 19 2017, 6:50 AM
Marostegui updated the task description. (Show Details)Jun 19 2017, 7:09 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-19T07:10:52Z] <marostegui> Deploy alter table s5 - codfw master - db2023 (and will replicate) so this will generate lag on codfw slaves - T166207

codfw master (db2023) has been finished, and now the changes are getting replicated to the slaves.

db2023:

root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb2023.codfw.wmnet wikidatawiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),



root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb2023.codfw.wmnet dewiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),

Mentioned in SAL (#wikimedia-operations) [2017-06-19T15:16:31Z] <marostegui> Deploy alter table labsdb1009 - T166207

Marostegui updated the task description. (Show Details)Jun 20 2017, 7:09 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-20T07:10:45Z] <marostegui> Deploy alter table s5 - db1095 - T166207

Change 360214 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1071

https://gerrit.wikimedia.org/r/360214

Change 360214 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1071

https://gerrit.wikimedia.org/r/360214

Mentioned in SAL (#wikimedia-operations) [2017-06-20T07:20:49Z] <marostegui> Deploy alter table s5 - db1071 - T166207

db1071 done:

root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1071 wikidatawiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1071 dewiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
Marostegui updated the task description. (Show Details)Jun 20 2017, 1:23 PM
Marostegui updated the task description. (Show Details)

For those tables existing on db1095, they are done:

root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1095 dewiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
ERROR 1146 (42S02) at line 1: Table 'dewiki.text' doesn't exist
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
ERROR 1146 (42S02) at line 1: Table 'dewiki.log_search' doesn't exist
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
ERROR 1146 (42S02) at line 1: Table 'dewiki.objectcache' doesn't exist
querycache_info
ERROR 1146 (42S02) at line 1: Table 'dewiki.querycache_info' doesn't exist
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
ERROR 1146 (42S02) at line 1: Table 'dewiki.transcache' doesn't exist
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1095 wikidatawiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
ERROR 1146 (42S02) at line 1: Table 'wikidatawiki.text' doesn't exist
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
ERROR 1146 (42S02) at line 1: Table 'wikidatawiki.log_search' doesn't exist
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
ERROR 1146 (42S02) at line 1: Table 'wikidatawiki.objectcache' doesn't exist
querycache_info
ERROR 1146 (42S02) at line 1: Table 'wikidatawiki.querycache_info' doesn't exist
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
ERROR 1146 (42S02) at line 1: Table 'wikidatawiki.transcache' doesn't exist
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),

Mentioned in SAL (#wikimedia-operations) [2017-06-20T13:26:52Z] <marostegui> Deploy alter table labsdb1010 - s5 - T166207

Mentioned in SAL (#wikimedia-operations) [2017-06-20T13:39:56Z] <marostegui> Deploy alter table on db1049 - s5 - T166207

Mentioned in SAL (#wikimedia-operations) [2017-06-20T13:47:56Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1071 - T166207 (duration: 00m 41s)

Change 360347 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1087

https://gerrit.wikimedia.org/r/360347

Change 360347 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1087

https://gerrit.wikimedia.org/r/360347

Mentioned in SAL (#wikimedia-operations) [2017-06-20T13:55:01Z] <marostegui> Deploy alter table db1087 - s5 - T166207

Mentioned in SAL (#wikimedia-operations) [2017-06-20T13:55:39Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1087 - T166207 (duration: 01m 41s)

Marostegui updated the task description. (Show Details)Jun 21 2017, 5:50 AM

db1049 is done:

root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1049 dewiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1049 wikidatawiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),

db1087 is done:

root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1087 dewiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1087 wikidatawiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),

Mentioned in SAL (#wikimedia-operations) [2017-06-21T05:54:51Z] <marostegui> Deploy alter table s5 - labsdb1011 - T166207

Mentioned in SAL (#wikimedia-operations) [2017-06-21T05:57:50Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1087 - T166207 (duration: 00m 44s)

Change 360605 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1082

https://gerrit.wikimedia.org/r/360605

Change 360605 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1082

https://gerrit.wikimedia.org/r/360605

Mentioned in SAL (#wikimedia-operations) [2017-06-21T06:04:41Z] <marostegui> Deploy alter table s5 - dbstore1002 - T166207

Mentioned in SAL (#wikimedia-operations) [2017-06-21T06:04:51Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1082 - T166207 (duration: 00m 44s)

Mentioned in SAL (#wikimedia-operations) [2017-06-21T06:05:41Z] <marostegui> Deploy alter table s5 - db1082 - T166207

db1082 is done:

root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1082 dewiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
root@neodymium:/home/marostegui#
root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1082 wikidatawiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
Marostegui updated the task description. (Show Details)Jun 21 2017, 8:23 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-21T08:27:51Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1082 - T166207 (duration: 00m 45s)

Change 360616 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1070

https://gerrit.wikimedia.org/r/360616

Change 360616 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1070

https://gerrit.wikimedia.org/r/360616

Mentioned in SAL (#wikimedia-operations) [2017-06-21T08:33:42Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1070 - T166207 (duration: 00m 44s)

Mentioned in SAL (#wikimedia-operations) [2017-06-21T08:34:38Z] <marostegui> Deploy alter table db1070 s5 - T166207

Marostegui updated the task description. (Show Details)Jun 21 2017, 10:32 AM

db1070 is done:

root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1070 dewiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1070 wikidatawiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
Marostegui updated the task description. (Show Details)Jun 21 2017, 1:03 PM

Mentioned in SAL (#wikimedia-operations) [2017-06-21T13:08:12Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1070 - T166207 (duration: 00m 46s)

Change 360645 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1045

https://gerrit.wikimedia.org/r/360645

Change 360645 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1045

https://gerrit.wikimedia.org/r/360645

Mentioned in SAL (#wikimedia-operations) [2017-06-21T13:14:26Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1045 - T166207 (duration: 00m 44s)

Mentioned in SAL (#wikimedia-operations) [2017-06-21T13:15:01Z] <marostegui> Deploy alter table s5 - db1045 - T166207

cloud-services-team I am going to deploy this alter table on labsdb1001 - so the s5 shard will get delayed there for around 24h or so.

Mentioned in SAL (#wikimedia-operations) [2017-06-21T13:16:31Z] <marostegui> Deploy alter table s5 - labsdb1001 - T166207

dbstore1002 is done:

root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdbstore1002 wikidatawiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdbstore1002 dewiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),

db1045 is done:

root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1045 dewiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1045 wikidatawiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
Marostegui updated the task description. (Show Details)Jun 22 2017, 7:08 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-22T07:12:25Z] <marostegui> Deploy alter table s5 - dbstore1001 - T166207

Mentioned in SAL (#wikimedia-operations) [2017-06-22T07:15:00Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1045 - T166207 (duration: 01m 03s)

Change 360799 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1026

https://gerrit.wikimedia.org/r/360799

Change 360799 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1026

https://gerrit.wikimedia.org/r/360799

Mentioned in SAL (#wikimedia-operations) [2017-06-22T07:21:42Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1026 - T166207 (duration: 00m 44s)

Mentioned in SAL (#wikimedia-operations) [2017-06-22T07:24:45Z] <marostegui> Deploy alter table s5 - db1026 - T166207

Marostegui updated the task description. (Show Details)Jun 22 2017, 9:07 AM

cloud-services-team labsdb1001 finished its alter table on s5. The delay was around 10h
I am going to run it on labsdb1003, I expect labsdb1003 to get more or less the same delay as labsdb1001, around 10-11h or so.

Mentioned in SAL (#wikimedia-operations) [2017-06-22T09:11:11Z] <marostegui> Deploy alter table s5 - labsdb1003 - T166207

db1026 is done:

root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1026 wikidatawiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdb1026 dewiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),

dbstore1001 is done:

root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdbstore1001 wikidatawiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
root@neodymium:/home/marostegui# for i in `cat s5_tables`; do echo $i; mysql --skip-ssl -hdbstore1001 dewiki -e "show create table $i\G" | grep PRIMAR;done
categorylinks
  PRIMARY KEY (`cl_from`,`cl_to`),
templatelinks
  PRIMARY KEY (`tl_from`,`tl_namespace`,`tl_title`),
pagelinks
  PRIMARY KEY (`pl_from`,`pl_namespace`,`pl_title`),
text
  PRIMARY KEY (`old_id`)
imagelinks
  PRIMARY KEY (`il_from`,`il_to`),
iwlinks
  PRIMARY KEY (`iwl_from`,`iwl_prefix`,`iwl_title`),
langlinks
  PRIMARY KEY (`ll_from`,`ll_lang`),
log_search
  PRIMARY KEY (`ls_field`,`ls_value`,`ls_log_id`),
math
  PRIMARY KEY (`math_inputhash`)
module_deps
  PRIMARY KEY (`md_module`,`md_skin`)
objectcache
  PRIMARY KEY (`keyname`),
querycache_info
  PRIMARY KEY (`qci_type`)
site_stats
  PRIMARY KEY (`ss_row_id`)
transcache
  PRIMARY KEY (`tc_url`)
user_former_groups
  PRIMARY KEY (`ufg_user`,`ufg_group`)
user_properties
  PRIMARY KEY (`up_user`,`up_property`),
Marostegui updated the task description. (Show Details)Jun 23 2017, 5:51 AM

Mentioned in SAL (#wikimedia-operations) [2017-06-23T05:59:29Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1026 - T166207 (duration: 00m 47s)

Marostegui closed this task as Resolved.Jun 23 2017, 8:04 AM
Marostegui updated the task description. (Show Details)

s5 is done

jcrespo reopened this task as Open.Jun 28 2017, 3:03 PM
This comment was removed by jcrespo.
jcrespo claimed this task.Jun 28 2017, 3:15 PM
This comment was removed by jcrespo.

labsdb1003 seems to be missing a primary key- T169038

Looks like it failed for dewiki (but not for wikidatawiki)

I am running alter now.

labsdb1001 is fine, so it might have been that punctual error, with so many alters to be done I probably missed it. Sorry for the inconveniences.
Thanks @jcrespo for taking care of it

Done, is this the right final state?

MariaDB [dewiki_p]> SHOW CREATE TABLE dewiki.categorylinks\G
*************************** 1. row ***************************
       Table: categorylinks
Create Table: CREATE TABLE `categorylinks` (
  `cl_from` int(8) unsigned NOT NULL DEFAULT '0',
  `cl_to` varbinary(255) NOT NULL DEFAULT '',
  `cl_sortkey` varbinary(230) NOT NULL DEFAULT '',
  `cl_timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  `cl_sortkey_prefix` varbinary(255) NOT NULL DEFAULT '',
  `cl_collation` varbinary(32) NOT NULL DEFAULT '',
  `cl_type` enum('page','subcat','file') NOT NULL DEFAULT 'page',
  PRIMARY KEY (`cl_from`,`cl_to`),
  KEY `cl_timestamp` (`cl_to`,`cl_timestamp`),
  KEY `cl_sortkey` (`cl_to`,`cl_type`,`cl_sortkey`,`cl_from`),
  KEY `cl_collation_ext` (`cl_collation`,`cl_to`,`cl_type`,`cl_from`)
) ENGINE=TokuDB DEFAULT CHARSET=binary `compression`='tokudb_zlib'
1 row in set (0.00 sec)

Yep! That is it!
Thanks!

jcrespo closed this task as Resolved.Jun 28 2017, 4:45 PM