Page MenuHomePhabricator

Schema change for refactored actor and comment storage
Open, MediumPublic

Description

From the checklist at https://wikitech.wikimedia.org/wiki/Schema_changes:

  1. ALTERs to run: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/482371/3/maintenance/archives/patch-drop-comment-fields.sql and https://gerrit.wikimedia.org/r/c/mediawiki/core/+/525605/8/maintenance/archives/patch-drop-user-fields.sql
  2. Where to run those changes: all.dblist
  3. When to run those changes: No time constraint.
  4. If the schema change is backwards compatible: Yes. The code no longer accesses these fields.
  5. If the schema change has been tested already on some of the test/beta wikis: It appears to have been auto-deployed to Beta. Things seem to be working, and if anyone has complained about things breaking I haven't heard it.
  6. If the data should be made available on the labs replicas and/or dumps: N/A, no new data is being added here. References to these fields from Labs replica views were removed in T212972 and T223406.

Progress:

  • s1
    • codfw
    • eqiad progress T233135#5691987
    • Run maintain-views on labs hosts to recreate the views (sudo /usr/local/sbin/maintain-views --databases jawiki --table protected_titles --replace-all)
  • s2
    • codfw
    • eqiad progress T233135#5578395
    • Run maintain-views on labs hosts to recreate the views
  • s3
    • codfw
    • eqiad progress T233135#5703954
    • Run maintain-views on labs hosts to recreate the views
  • s4
    • codfw
    • eqiad
    • Run maintain-views on labs hosts to recreate the views
  • s5
    • codfw
    • eqiad T233135#5655648
    • Run maintain-views on labs hosts to recreate the views
  • s7
    • codfw
    • eqiad progress T233135#5571998
    • Run maintain-views on labs hosts to recreate the views
  • s8
    • codfw
    • eqiad T233135#5671879
    • Run maintain-views on labs hosts to recreate the views
  • labswiki
  • labtestwiki
  • Remove columns from modules/role/files/mariadb/filtered_tables.txt once everything is done.

Details

Related Gerrit Patches:
operations/puppet : productionindex-conf.yaml: Remove unused index

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Mentioned in SAL (#wikimedia-operations) [2019-10-03T06:16:03Z] <marostegui> Deploy schema change on db2089:3316 T233135 T234066

This is how the tables look like after this alter and the alter for the special slaves (T233625)

root@cumin1001:/home/marostegui# for i in archive ipblocks image oldimage filearchive recentchanges logging protected_titles; do echo $i; mysql.py -hdb2089:3316 frwiki -e "show create table $i\G";done
archive
*************************** 1. row ***************************
       Table: archive
Create Table: CREATE TABLE `archive` (
  `ar_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `ar_namespace` int(11) NOT NULL DEFAULT '0',
  `ar_title` varbinary(255) NOT NULL DEFAULT '',
  `ar_comment_id` bigint(20) unsigned NOT NULL,
  `ar_actor` bigint(20) unsigned NOT NULL,
  `ar_timestamp` varbinary(14) NOT NULL DEFAULT '',
  `ar_minor_edit` tinyint(1) NOT NULL DEFAULT '0',
  `ar_rev_id` int(10) unsigned NOT NULL,
  `ar_text_id` int(10) unsigned NOT NULL DEFAULT '0',
  `ar_deleted` tinyint(1) unsigned NOT NULL DEFAULT '0',
  `ar_len` int(8) unsigned DEFAULT NULL,
  `ar_page_id` int(10) unsigned DEFAULT NULL,
  `ar_parent_id` int(10) unsigned DEFAULT NULL,
  `ar_sha1` varbinary(32) NOT NULL DEFAULT '',
  `ar_content_model` varbinary(32) DEFAULT NULL,
  `ar_content_format` varbinary(64) DEFAULT NULL,
  PRIMARY KEY (`ar_id`),
  UNIQUE KEY `ar_revid_uniq` (`ar_rev_id`),
  KEY `name_title_timestamp` (`ar_namespace`,`ar_title`,`ar_timestamp`),
  KEY `ar_actor_timestamp` (`ar_actor`,`ar_timestamp`)
) ENGINE=InnoDB AUTO_INCREMENT=10493733 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
ipblocks
*************************** 1. row ***************************
       Table: ipblocks
Create Table: CREATE TABLE `ipblocks` (
  `ipb_id` int(8) NOT NULL AUTO_INCREMENT,
  `ipb_address` tinyblob NOT NULL,
  `ipb_user` int(8) unsigned NOT NULL DEFAULT '0',
  `ipb_reason_id` bigint(20) unsigned NOT NULL,
  `ipb_timestamp` varbinary(14) NOT NULL DEFAULT '',
  `ipb_auto` tinyint(1) NOT NULL DEFAULT '0',
  `ipb_anon_only` tinyint(1) NOT NULL DEFAULT '0',
  `ipb_create_account` tinyint(1) NOT NULL DEFAULT '1',
  `ipb_expiry` varbinary(14) NOT NULL DEFAULT '',
  `ipb_range_start` tinyblob NOT NULL,
  `ipb_range_end` tinyblob NOT NULL,
  `ipb_enable_autoblock` tinyint(1) NOT NULL DEFAULT '1',
  `ipb_deleted` tinyint(1) NOT NULL DEFAULT '0',
  `ipb_block_email` tinyint(1) NOT NULL DEFAULT '0',
  `ipb_by_actor` bigint(20) unsigned NOT NULL,
  `ipb_allow_usertalk` tinyint(1) NOT NULL DEFAULT '1',
  `ipb_parent_block_id` int(11) DEFAULT NULL,
  `ipb_sitewide` tinyint(1) NOT NULL DEFAULT '1',
  PRIMARY KEY (`ipb_id`),
  UNIQUE KEY `ipb_address_unique` (`ipb_address`(255),`ipb_user`,`ipb_auto`),
  KEY `ipb_user` (`ipb_user`),
  KEY `ipb_timestamp` (`ipb_timestamp`),
  KEY `ipb_expiry` (`ipb_expiry`),
  KEY `ipb_range` (`ipb_range_start`(20),`ipb_range_end`(20)),
  KEY `ipb_parent_block_id` (`ipb_parent_block_id`)
) ENGINE=InnoDB AUTO_INCREMENT=3976644 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
image
*************************** 1. row ***************************
       Table: image
Create Table: CREATE TABLE `image` (
  `img_name` varbinary(255) NOT NULL DEFAULT '',
  `img_size` int(8) unsigned NOT NULL DEFAULT '0',
  `img_width` int(5) NOT NULL DEFAULT '0',
  `img_height` int(5) NOT NULL DEFAULT '0',
  `img_metadata` mediumblob NOT NULL,
  `img_bits` int(3) NOT NULL DEFAULT '0',
  `img_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
  `img_major_mime` enum('unknown','application','audio','image','text','video','message','model','multipart') NOT NULL DEFAULT 'unknown',
  `img_minor_mime` varbinary(100) NOT NULL DEFAULT 'unknown',
  `img_description_id` bigint(20) unsigned NOT NULL,
  `img_actor` bigint(20) unsigned NOT NULL,
  `img_timestamp` varbinary(14) NOT NULL DEFAULT '',
  `img_sha1` varbinary(32) NOT NULL DEFAULT '',
  `img_deleted` tinyint(3) unsigned NOT NULL DEFAULT '0',
  PRIMARY KEY (`img_name`),
  KEY `img_size` (`img_size`),
  KEY `img_timestamp` (`img_timestamp`),
  KEY `img_media_mime` (`img_media_type`,`img_major_mime`,`img_minor_mime`),
  KEY `img_actor_timestamp` (`img_actor`,`img_timestamp`),
  KEY `img_sha1` (`img_sha1`(10))
) ENGINE=InnoDB DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
oldimage
*************************** 1. row ***************************
       Table: oldimage
Create Table: CREATE TABLE `oldimage` (
  `oi_name` varbinary(255) NOT NULL DEFAULT '',
  `oi_archive_name` varbinary(255) NOT NULL DEFAULT '',
  `oi_size` int(8) unsigned NOT NULL DEFAULT '0',
  `oi_width` int(5) NOT NULL DEFAULT '0',
  `oi_height` int(5) NOT NULL DEFAULT '0',
  `oi_bits` int(3) NOT NULL DEFAULT '0',
  `oi_description_id` bigint(20) unsigned NOT NULL,
  `oi_actor` bigint(20) unsigned NOT NULL,
  `oi_timestamp` varbinary(14) NOT NULL DEFAULT '',
  `oi_sha1` varbinary(32) NOT NULL DEFAULT '',
  `oi_metadata` mediumblob NOT NULL,
  `oi_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
  `oi_major_mime` enum('unknown','application','audio','image','text','video','message','model','multipart') NOT NULL DEFAULT 'unknown',
  `oi_minor_mime` varbinary(100) NOT NULL DEFAULT 'unknown',
  `oi_deleted` tinyint(3) unsigned NOT NULL DEFAULT '0',
  KEY `oi_name_timestamp` (`oi_name`,`oi_timestamp`),
  KEY `oi_name_archive_name` (`oi_name`,`oi_archive_name`(14)),
  KEY `oi_actor_timestamp` (`oi_actor`,`oi_timestamp`),
  KEY `oi_sha1` (`oi_sha1`(10))
) ENGINE=InnoDB DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
filearchive
*************************** 1. row ***************************
       Table: filearchive
Create Table: CREATE TABLE `filearchive` (
  `fa_id` int(11) NOT NULL AUTO_INCREMENT,
  `fa_name` varbinary(255) NOT NULL DEFAULT '',
  `fa_archive_name` varbinary(255) DEFAULT '',
  `fa_storage_group` varbinary(16) DEFAULT NULL,
  `fa_storage_key` varbinary(64) DEFAULT '',
  `fa_deleted_user` int(11) DEFAULT NULL,
  `fa_deleted_timestamp` varbinary(14) DEFAULT '',
  `fa_deleted_reason_id` bigint(20) unsigned NOT NULL,
  `fa_size` int(8) unsigned DEFAULT '0',
  `fa_width` int(5) DEFAULT '0',
  `fa_height` int(5) DEFAULT '0',
  `fa_metadata` mediumblob,
  `fa_bits` int(3) DEFAULT '0',
  `fa_media_type` enum('UNKNOWN','BITMAP','DRAWING','AUDIO','VIDEO','MULTIMEDIA','OFFICE','TEXT','EXECUTABLE','ARCHIVE','3D') DEFAULT NULL,
  `fa_major_mime` enum('unknown','application','audio','image','text','video','message','model','multipart') DEFAULT 'unknown',
  `fa_minor_mime` varbinary(100) DEFAULT 'unknown',
  `fa_description_id` bigint(20) unsigned NOT NULL,
  `fa_actor` bigint(20) unsigned NOT NULL,
  `fa_timestamp` varbinary(14) DEFAULT '',
  `fa_deleted` tinyint(1) unsigned NOT NULL DEFAULT '0',
  `fa_sha1` varbinary(32) NOT NULL DEFAULT '',
  PRIMARY KEY (`fa_id`),
  KEY `fa_name` (`fa_name`,`fa_timestamp`),
  KEY `fa_storage_group` (`fa_storage_group`,`fa_storage_key`),
  KEY `fa_deleted_timestamp` (`fa_deleted_timestamp`),
  KEY `fa_sha1` (`fa_sha1`(10)),
  KEY `fa_actor_timestamp` (`fa_actor`,`fa_timestamp`)
) ENGINE=InnoDB AUTO_INCREMENT=140921 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
recentchanges
*************************** 1. row ***************************
       Table: recentchanges
Create Table: CREATE TABLE `recentchanges` (
  `rc_id` int(8) NOT NULL AUTO_INCREMENT,
  `rc_timestamp` varbinary(14) NOT NULL DEFAULT '',
  `rc_actor` bigint(20) unsigned NOT NULL,
  `rc_namespace` int(11) NOT NULL DEFAULT '0',
  `rc_title` varbinary(255) NOT NULL DEFAULT '',
  `rc_comment_id` bigint(20) unsigned NOT NULL,
  `rc_minor` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `rc_bot` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `rc_new` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `rc_cur_id` int(10) unsigned NOT NULL DEFAULT '0',
  `rc_this_oldid` int(10) unsigned NOT NULL DEFAULT '0',
  `rc_last_oldid` int(10) unsigned NOT NULL DEFAULT '0',
  `rc_type` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `rc_source` varbinary(16) NOT NULL DEFAULT '',
  `rc_patrolled` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `rc_ip` varbinary(40) NOT NULL DEFAULT '',
  `rc_old_len` int(10) DEFAULT NULL,
  `rc_new_len` int(10) DEFAULT NULL,
  `rc_deleted` tinyint(1) unsigned NOT NULL DEFAULT '0',
  `rc_logid` int(10) unsigned NOT NULL DEFAULT '0',
  `rc_log_type` varbinary(255) DEFAULT NULL,
  `rc_log_action` varbinary(255) DEFAULT NULL,
  `rc_params` blob,
  PRIMARY KEY (`rc_id`),
  KEY `rc_timestamp` (`rc_timestamp`),
  KEY `rc_cur_id` (`rc_cur_id`),
  KEY `new_name_timestamp` (`rc_new`,`rc_namespace`,`rc_timestamp`),
  KEY `rc_ip` (`rc_ip`),
  KEY `tmp_2` (`rc_bot`,`rc_timestamp`),
  KEY `tmp_3` (`rc_namespace`,`rc_timestamp`),
  KEY `rc_name_type_patrolled_timestamp` (`rc_namespace`,`rc_type`,`rc_patrolled`,`rc_timestamp`),
  KEY `rc_ns_actor` (`rc_namespace`,`rc_actor`),
  KEY `rc_actor` (`rc_actor`,`rc_timestamp`),
  KEY `rc_namespace_title_timestamp` (`rc_namespace`,`rc_title`,`rc_timestamp`),
  KEY `rc_this_oldid` (`rc_this_oldid`)
) ENGINE=InnoDB AUTO_INCREMENT=395253359 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
logging
*************************** 1. row ***************************
       Table: logging
Create Table: CREATE TABLE `logging` (
  `log_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `log_type` varbinary(32) NOT NULL DEFAULT '',
  `log_action` varbinary(32) NOT NULL DEFAULT '',
  `log_timestamp` varbinary(14) NOT NULL DEFAULT '19700101000000',
  `log_namespace` int(11) NOT NULL DEFAULT '0',
  `log_title` varbinary(255) NOT NULL DEFAULT '',
  `log_comment_id` bigint(20) unsigned NOT NULL,
  `log_params` blob NOT NULL,
  `log_deleted` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `log_actor` bigint(20) unsigned NOT NULL,
  `log_page` int(10) unsigned DEFAULT NULL,
  PRIMARY KEY (`log_id`),
  KEY `type_time` (`log_type`,`log_timestamp`),
  KEY `page_time` (`log_namespace`,`log_title`,`log_timestamp`),
  KEY `times` (`log_timestamp`),
  KEY `log_page_id_time` (`log_page`,`log_timestamp`),
  KEY `type_action` (`log_type`,`log_action`,`log_timestamp`),
  KEY `actor_time` (`log_actor`,`log_timestamp`),
  KEY `log_actor_type_time` (`log_actor`,`log_type`,`log_timestamp`),
  KEY `log_type_action` (`log_type`,`log_action`,`log_timestamp`)
) ENGINE=InnoDB AUTO_INCREMENT=102243986 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
protected_titles
*************************** 1. row ***************************
       Table: protected_titles
Create Table: CREATE TABLE `protected_titles` (
  `pt_namespace` int(11) NOT NULL DEFAULT '0',
  `pt_title` varbinary(255) NOT NULL DEFAULT '',
  `pt_user` int(10) unsigned NOT NULL DEFAULT '0',
  `pt_reason_id` bigint(20) unsigned NOT NULL,
  `pt_timestamp` varbinary(14) NOT NULL DEFAULT '',
  `pt_expiry` varbinary(14) NOT NULL DEFAULT '',
  `pt_create_perm` varbinary(60) NOT NULL DEFAULT '',
  PRIMARY KEY (`pt_namespace`,`pt_title`),
  KEY `pt_timestamp` (`pt_timestamp`)
) ENGINE=InnoDB DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8

Mentioned in SAL (#wikimedia-operations) [2019-10-03T08:26:52Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1096:3316 for schema change T233135 T234066', diff saved to https://phabricator.wikimedia.org/P9236 and previous config saved to /var/cache/conftool/dbconfig/20191003-082651-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2019-10-03T08:33:13Z] <marostegui> Deploy schema change on db2087:3316 T233135 T234066

Anomie added a comment.Oct 3 2019, 3:51 PM

This is how the tables look like after this alter and the alter for the special slaves (T233625)

Those all look good as far as this task goes. Thanks!

I see several unrelated schema differences versus tables.sql, which isn't too surprising. We already know about most of them, I think.

This is how the tables look like after this alter and the alter for the special slaves (T233625)

Those all look good as far as this task goes. Thanks!
I see several unrelated schema differences versus tables.sql, which isn't too surprising. We already know about most of them, I think.

Yeah, to be tackle in separate tasks, we still have lots of work ahead of us to make ourselves consistent with tables.sql

Mentioned in SAL (#wikimedia-operations) [2019-10-04T06:16:47Z] <marostegui> Deploy schema change on dbstore1005:3316 T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-04T06:40:59Z] <marostegui> Deploy schema change on db2114 T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-04T08:41:38Z] <marostegui> Deploy schema change on db2076 (sanitarium master) with replication T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-04T12:28:29Z] <marostegui> Deploy schema change on db2097:3316 T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-04T14:03:21Z] <marostegui> Deploy schema change on db2117 T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-07T04:54:12Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1098:3316 for schema change T233135 T234066', diff saved to https://phabricator.wikimedia.org/P9245 and previous config saved to /var/cache/conftool/dbconfig/20191007-045411-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2019-10-07T05:25:27Z] <marostegui> Deploy schema change on db2124 T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-07T09:55:01Z] <marostegui> Deploy schema change on db2129 (s6 codfw master), this will generate lag on s6 codfw - T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-07T13:17:22Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1098:3316 schema change T233135 T234066', diff saved to https://phabricator.wikimedia.org/P9248 and previous config saved to /var/cache/conftool/dbconfig/20191007-131720-marostegui.json

Marostegui updated the task description. (Show Details)Oct 7 2019, 2:12 PM

Mentioned in SAL (#wikimedia-operations) [2019-10-07T14:17:40Z] <marostegui> Deploy schema change on db1139:3316 - T233135 T234066

The only view that needs to be recreated on labs hosts is the one for protected_titles the rest are fine.

for i in archive ipblocks image oldimage filearchive recentchanges logging protected_titles logging; do echo $i; mysql.py -hlabsdb1009 frwiki_p -e "select * from $i limit 1;" | grep ERROR;done
archive
ipblocks
image
oldimage
filearchive
recentchanges
logging
protected_titles
ERROR 1356 (HY000) at line 1: View 'frwiki_p.protected_titles' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them
logging
Marostegui updated the task description. (Show Details)Oct 9 2019, 9:01 AM
Anomie added a comment.Oct 9 2019, 2:56 PM

Good catch. That's because protected_titles is in "fullviews" instead of "customviews".

Mentioned in SAL (#wikimedia-operations) [2019-10-10T04:53:54Z] <marostegui> Deploy schema change on db1061 (s6 eqiad master) - T233135 T234066

Marostegui updated the task description. (Show Details)Oct 10 2019, 10:11 AM

Mentioned in SAL (#wikimedia-operations) [2019-10-10T11:40:53Z] <marostegui> Deploy schema change on s7 codfw master (db2118), this will generate lag on s7 codfw - T234066 T233135

Marostegui updated the task description. (Show Details)Oct 11 2019, 4:51 AM
Marostegui added a comment.EditedOct 14 2019, 9:13 AM

s7 eqiad progress

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1003
  • db1136
  • db1125
  • db1116
  • db1101
  • db1098
  • db1094
  • db1090
  • db1086
  • db1079
  • db1062

Mentioned in SAL (#wikimedia-operations) [2019-10-14T14:21:34Z] <marostegui> Deploy schema change on db1116:3317 T234066 T233135

Mentioned in SAL (#wikimedia-operations) [2019-10-15T05:28:49Z] <marostegui> Deploy schema change on db1098:3317 T234066 T233135

Mentioned in SAL (#wikimedia-operations) [2019-10-16T05:08:05Z] <marostegui> Deploy schema change on s7 sanitarium master (db1079) this will create lag on s7 labsdb T233135 T234066

Marostegui added a comment.EditedOct 16 2019, 5:16 AM

s2 eqiad progress

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1004
  • db1129
  • db1125
  • db1122
  • db1105
  • db1103
  • db1095
  • db1090
  • db1076
  • db1074

Mentioned in SAL (#wikimedia-operations) [2019-10-16T05:18:28Z] <marostegui> Deploy schema change on s2 sanitarium master (db1074) this will create lag on s2 labsdb T233135 T234066

Marostegui updated the task description. (Show Details)Oct 16 2019, 5:56 AM

Mentioned in SAL (#wikimedia-operations) [2019-10-16T10:17:10Z] <marostegui> Stop replication on s2 codfw master for schema change and to modify sanitarium triggers T234066 T233135 T234704

Mentioned in SAL (#wikimedia-operations) [2019-10-16T15:09:17Z] <marostegui> Recreate views for protected_titles on s2 and s7 on labsdb1009 and labsdb1012 - T233135

Mentioned in SAL (#wikimedia-operations) [2019-10-16T15:17:27Z] <marostegui> Deploy schema change on dbstore1004:3312 - T234066 T233135

Marostegui updated the task description. (Show Details)Oct 17 2019, 5:04 AM
Marostegui updated the task description. (Show Details)Oct 17 2019, 5:25 AM
Marostegui updated the task description. (Show Details)Oct 17 2019, 5:30 AM

Mentioned in SAL (#wikimedia-operations) [2019-10-18T05:15:54Z] <marostegui> Deploy schema change on db1129 T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-10-21T05:09:38Z] <marostegui> Deploy schema change on s7 primary master db1062 - T234066 T233135

Mentioned in SAL (#wikimedia-operations) [2019-10-21T05:14:28Z] <marostegui> Deploy schema change on db1090:3312 T234066 T233135

Marostegui updated the task description. (Show Details)Oct 21 2019, 9:25 AM

Mentioned in SAL (#wikimedia-operations) [2019-10-21T13:04:47Z] <marostegui> Deploy schema change on db1122 (s2 primary master) - T233135 T234066

Nuria added a project: Analytics.EditedOct 21 2019, 7:50 PM
Nuria added subscribers: JAllemandou, Milimetric, Nuria.

Pinging analytics temporarily so we know these changes are happening, it should not affect scooping of these tables cause i think we generate schema every time we scoop but want to make sure. cc @Milimetric @JAllemandou In terms of algorithms we have not used these fields for a while.

Marostegui updated the task description. (Show Details)Oct 22 2019, 4:58 AM

Pinging analytics temporarily so we know these changes are happening, it shoudl not affect scooping of these tables cause i think we generate schema everytime we scoop but want to make sure. cc @Milimetric @JAllemandou In terms of algorithms we have not used these fields for a while.

I confirm the fields are not used in our current sqoop queries. The change should be transparent for us.

Nuria removed a subscriber: Nuria.
jcrespo mentioned this in Unknown Object (Task).Nov 6 2019, 10:10 AM

Mentioned in SAL (#wikimedia-operations) [2019-11-12T06:40:51Z] <marostegui> Deploy schema change on s5 codfw with replication, this will generate lag on s5 codfw T233135 T234066

Marostegui updated the task description. (Show Details)Tue, Nov 12, 10:32 AM
Marostegui added a comment.EditedTue, Nov 12, 11:35 AM

s5 eqiad progress

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1003
  • db1130
  • db1124
  • db1113
  • db1110
  • db1102
  • db1100
  • db1097
  • db1096
  • db1082

Mentioned in SAL (#wikimedia-operations) [2019-11-12T15:41:29Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1097:3315 for a schema change T233135 T234066', diff saved to https://phabricator.wikimedia.org/P9600 and previous config saved to /var/cache/conftool/dbconfig/20191112-154127-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2019-11-12T15:49:26Z] <marostegui> Deploy schema change on db1102:3315 T233135 T234066

Marostegui updated the task description. (Show Details)Wed, Nov 13, 8:06 AM
Marostegui updated the task description. (Show Details)

Mentioned in SAL (#wikimedia-operations) [2019-11-14T08:41:30Z] <marostegui> Deploy schema change with replication on db1082, this will generate lag on s5 labs - T233135 T234066

Mentioned in SAL (#wikimedia-operations) [2019-11-15T05:57:28Z] <marostegui> Run maintain-views for s5 on labsdb1009, labsdb1010, labsdb1012 (pending labsdb1011 as it is still running the schema change) T233135

Mentioned in SAL (#wikimedia-operations) [2019-11-15T10:45:27Z] <marostegui> Run maintain-views for s5 on labsdb1011 T233135

Marostegui updated the task description. (Show Details)Fri, Nov 15, 10:46 AM

Mentioned in SAL (#wikimedia-operations) [2019-11-18T05:53:57Z] <marostegui> Deploy schema change on s5 primary master db1100 - T233135 T234066

Marostegui updated the task description. (Show Details)Mon, Nov 18, 9:09 AM

Mentioned in SAL (#wikimedia-operations) [2019-11-18T09:11:12Z] <marostegui> Deploy schema change on s8 codfw, this will generate lag on s8 codfw - T233135 T234066

Marostegui updated the task description. (Show Details)Mon, Nov 18, 2:18 PM
Marostegui added a comment.EditedMon, Nov 18, 2:23 PM

s8 eqiad progress

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1005
  • db1126
  • db1124
  • db1116
  • db1109
  • db1104
  • db1101
  • db1099
  • db1092
  • db1087
Marostegui updated the task description. (Show Details)Tue, Nov 19, 5:28 AM

Mentioned in SAL (#wikimedia-operations) [2019-11-25T14:37:26Z] <marostegui> Deploy schema change on s1 codfw (this will generate lag on codfw) - T234066 T233135

Marostegui updated the task description. (Show Details)Tue, Nov 26, 5:47 AM
Marostegui added a comment.EditedTue, Nov 26, 5:50 AM

s1 eqiad progress

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1003
  • db1139
  • db1134
  • db1124
  • db1119
  • db1118
  • db1114
  • db1106
  • db1105
  • db1099
  • db1089
  • db1083
  • db1080

Mentioned in SAL (#wikimedia-operations) [2019-11-26T09:18:36Z] <marostegui> Run maintain-views for wikidatawiki.protected_title view on labsdb hosts T233135

Mentioned in SAL (#wikimedia-operations) [2019-11-26T09:26:50Z] <marostegui> Deploy schema change on s8 primary master (db1109) - T234066 T233135 T237120

Marostegui updated the task description. (Show Details)Tue, Nov 26, 11:33 AM
Marostegui updated the task description. (Show Details)Wed, Nov 27, 7:44 AM

Mentioned in SAL (#wikimedia-operations) [2019-11-28T13:37:03Z] <marostegui> Recreate views for enwiki_p.protected_titles for all labsdb hosts - T233135

Mentioned in SAL (#wikimedia-operations) [2019-11-28T13:37:26Z] <marostegui> Deploy schema change on db1106 with replication (lag will appear on s1 on labs) - T234066 T233135

Marostegui updated the task description. (Show Details)Thu, Nov 28, 1:40 PM
Marostegui updated the task description. (Show Details)Fri, Nov 29, 9:20 AM
Marostegui updated the task description. (Show Details)Fri, Nov 29, 6:25 PM
Marostegui added a comment.EditedMon, Dec 2, 5:55 AM

s3 eqiad progress

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1004
  • db1124
  • db1123
  • db1112
  • db1095
  • db1078
  • db1075
Marostegui updated the task description. (Show Details)Mon, Dec 2, 6:25 AM
Marostegui updated the task description. (Show Details)Wed, Dec 4, 5:57 AM
Marostegui updated the task description. (Show Details)Thu, Dec 5, 5:43 AM