Page MenuHomePhabricator

Schema change: Make page.page_restrictions column NULL
Closed, ResolvedPublic

Description

From https://wikitech.wikimedia.org/wiki/Schema_changes#Workflow_of_a_schema_change:

10.4 hosts with the tinyblob version need: alter table page change column page_restrictions page_restrictions tinyblob default NULL; See T248333#6004421

Schema change progress:

  • s5 All varbinary except: enwikivoyage tinyblob
  • wikitech (labswiki)
  • labstestwiki @Andrew

Event Timeline

TK-999 created this task.Mar 23 2020, 5:53 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 23 2020, 5:53 PM
Anomie added a subscriber: Anomie.Mar 23 2020, 6:00 PM
Krinkle updated the task description. (Show Details)Mar 23 2020, 9:51 PM
Krinkle updated the task description. (Show Details)
Marostegui renamed this task from Review schema changes for T218446 to Schema change: Make page.page_restrictions column NULL.Mar 24 2020, 6:04 AM
Marostegui triaged this task as Medium priority.
Marostegui moved this task from Triage to Next on the DBA board.
Marostegui updated the task description. (Show Details)
Marostegui moved this task from Backlog to In progress on the Blocked-on-schema-change board.
Marostegui moved this task from Next to In progress on the DBA board.
Marostegui added a subscriber: Andrew.
Marostegui added a comment.EditedMar 24 2020, 11:12 AM

I will start with s6, and first, host by host in codfw, to make sure nothing unexpected happens - also note that this schema change will require all the masters to be switched over, as it is changing the column data type, and that operation cannot be done online:

s6 progress:

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1005
  • db2129
  • db2124
  • db2117
  • db2114
  • db2097
  • db2095
  • db2089
  • db2087
  • db2076
  • db1139
  • db1131
  • db1125
  • db1113
  • db1098
  • db1096
  • db1093
  • db1088
  • db1085

How the tables look like after the alter @TK-999 please confirm this looks good:

root@cumin1001:/home/marostegui/T248333# for i in frwiki jawiki ruwiki; do echo $i; mysql.py -hdb2076 $i -e "show create table page\G"; done
frwiki
*************************** 1. row ***************************
       Table: page
Create Table: CREATE TABLE `page` (
  `page_id` int(8) unsigned NOT NULL AUTO_INCREMENT,
  `page_namespace` int(11) NOT NULL DEFAULT '0',
  `page_title` varbinary(255) NOT NULL DEFAULT '',
  `page_restrictions` tinyblob,
  `page_is_redirect` tinyint(1) unsigned NOT NULL DEFAULT '0',
  `page_is_new` tinyint(1) unsigned NOT NULL DEFAULT '0',
  `page_random` double unsigned NOT NULL DEFAULT '0',
  `page_touched` varbinary(14) NOT NULL DEFAULT '',
  `page_links_updated` varbinary(14) DEFAULT NULL,
  `page_latest` int(8) unsigned NOT NULL DEFAULT '0',
  `page_len` int(8) unsigned NOT NULL DEFAULT '0',
  `page_content_model` varbinary(32) DEFAULT NULL,
  `page_lang` varbinary(35) DEFAULT NULL,
  PRIMARY KEY (`page_id`),
  UNIQUE KEY `name_title` (`page_namespace`,`page_title`),
  KEY `page_random` (`page_random`),
  KEY `page_len` (`page_len`),
  KEY `page_redirect_namespace_len` (`page_is_redirect`,`page_namespace`,`page_len`)
) ENGINE=InnoDB AUTO_INCREMENT=13203343 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
jawiki
*************************** 1. row ***************************
       Table: page
Create Table: CREATE TABLE `page` (
  `page_id` int(8) unsigned NOT NULL AUTO_INCREMENT,
  `page_namespace` int(11) NOT NULL DEFAULT '0',
  `page_title` varbinary(255) NOT NULL DEFAULT '',
  `page_restrictions` tinyblob,
  `page_is_redirect` tinyint(1) unsigned NOT NULL DEFAULT '0',
  `page_is_new` tinyint(1) unsigned NOT NULL DEFAULT '0',
  `page_random` double unsigned NOT NULL DEFAULT '0',
  `page_touched` varbinary(14) NOT NULL DEFAULT '',
  `page_links_updated` varbinary(14) DEFAULT NULL,
  `page_latest` int(8) unsigned NOT NULL DEFAULT '0',
  `page_len` int(8) unsigned NOT NULL DEFAULT '0',
  `page_content_model` varbinary(32) DEFAULT NULL,
  `page_lang` varbinary(35) DEFAULT NULL,
  PRIMARY KEY (`page_id`),
  UNIQUE KEY `name_title` (`page_namespace`,`page_title`),
  KEY `page_random` (`page_random`),
  KEY `page_len` (`page_len`),
  KEY `page_redirect_namespace_len` (`page_is_redirect`,`page_namespace`,`page_len`)
) ENGINE=InnoDB AUTO_INCREMENT=4104185 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
ruwiki
*************************** 1. row ***************************
       Table: page
Create Table: CREATE TABLE `page` (
  `page_id` int(8) unsigned NOT NULL AUTO_INCREMENT,
  `page_namespace` int(11) NOT NULL DEFAULT '0',
  `page_title` varbinary(255) NOT NULL DEFAULT '',
  `page_restrictions` tinyblob,
  `page_is_redirect` tinyint(1) unsigned NOT NULL DEFAULT '0',
  `page_is_new` tinyint(1) unsigned NOT NULL DEFAULT '0',
  `page_random` double unsigned NOT NULL DEFAULT '0',
  `page_touched` varbinary(14) NOT NULL DEFAULT '',
  `page_links_updated` varbinary(14) DEFAULT NULL,
  `page_latest` int(8) unsigned NOT NULL DEFAULT '0',
  `page_len` int(8) unsigned NOT NULL DEFAULT '0',
  `page_content_model` varbinary(32) DEFAULT NULL,
  `page_lang` varbinary(35) DEFAULT NULL,
  PRIMARY KEY (`page_id`),
  UNIQUE KEY `name_title` (`page_namespace`,`page_title`),
  KEY `page_random` (`page_random`),
  KEY `page_len` (`page_len`),
  KEY `page_redirect_namespace_len` (`page_is_redirect`,`page_namespace`,`page_len`)
) ENGINE=InnoDB AUTO_INCREMENT=8253087 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8

It is looking good, thank you!

Anomie added a comment.EditedMar 24 2020, 1:23 PM

as it is changing the column data type

It shouldn't be changing the type, only changing from NOT NULL to NULL. It looks like that's still expensive, although perhaps less so.

Ah, fun, schema drift. Looks like 149 wikis have tinyblob matching tables.sql, while 794 have varbinary(255) instead.

If you want to, it would be fine to leave the ones that are varbinary(255) as such as long as the NOT NULL is removed. Actually it would even be fine to just set DEFAULT '' for those. We hope to drop the column entirely in the not-too-distant future, so drift now doesn't matter much.

as it is changing the column data type

It shouldn't be changing the type, only changing from NOT NULL to NULL. It looks like that's still expensive, although perhaps less so.

Ah, fun, schema drift. Looks like 149 wikis have tinyblob matching tables.sql, while 794 have varbinary(255) instead.

Ah, our beloved schema drifts :-(

If you want to, it would be fine to leave the ones that are varbinary(255) as such as long as the NOT NULL is removed. Actually it would probably even be fine to just set DEFAULT '' for those. We hope to drop the column entirely in the not-too-distant future, so drift now doesn't matter much.

We could leave the masters as varbinary(255) and just do a ' DEFAULT ''`, that won't lock the table. That way we can do the slaves quicker, and just make sure the masters are not getting any data type changes. Otherwise scanning all the wikis and adapting on the fly can be just painful as I am not sure even if all the host might have the same drifts - not the same time I see drifts within the same section.

So for those with varbinary we can just:

alter table page change page_restrictions page_restrictions varbinary(255) DEFAULT ''

I have confirmed that changing the DEFAULT ' ' doesn't produce any locking - this is how db2114 looks like

root@cumin1001:/home/marostegui/T248333# for i in frwiki jawiki ruwiki; do echo $i; mysql.py -hdb2114 $i -e "show create table page\G"; done
frwiki
*************************** 1. row ***************************
       Table: page
Create Table: CREATE TABLE `page` (
  `page_id` int(8) unsigned NOT NULL AUTO_INCREMENT,
  `page_namespace` int(11) NOT NULL DEFAULT 0,
  `page_title` varbinary(255) NOT NULL DEFAULT '',
  `page_restrictions` varbinary(255) DEFAULT '',
  `page_is_redirect` tinyint(1) unsigned NOT NULL DEFAULT 0,
  `page_is_new` tinyint(1) unsigned NOT NULL DEFAULT 0,
  `page_random` double unsigned NOT NULL DEFAULT 0,
  `page_touched` varbinary(14) NOT NULL DEFAULT '',
  `page_links_updated` varbinary(14) DEFAULT NULL,
  `page_latest` int(8) unsigned NOT NULL DEFAULT 0,
  `page_len` int(8) unsigned NOT NULL DEFAULT 0,
  `page_content_model` varbinary(32) DEFAULT NULL,
  `page_lang` varbinary(35) DEFAULT NULL,
  PRIMARY KEY (`page_id`),
  UNIQUE KEY `name_title` (`page_namespace`,`page_title`),
  KEY `page_random` (`page_random`),
  KEY `page_len` (`page_len`),
  KEY `page_redirect_namespace_len` (`page_is_redirect`,`page_namespace`,`page_len`)
) ENGINE=InnoDB AUTO_INCREMENT=13203665 DEFAULT CHARSET=binary
jawiki
*************************** 1. row ***************************
       Table: page
Create Table: CREATE TABLE `page` (
  `page_id` int(8) unsigned NOT NULL AUTO_INCREMENT,
  `page_namespace` int(11) NOT NULL DEFAULT 0,
  `page_title` varbinary(255) NOT NULL DEFAULT '',
  `page_restrictions` varbinary(255) DEFAULT '',
  `page_is_redirect` tinyint(1) unsigned NOT NULL DEFAULT 0,
  `page_is_new` tinyint(1) unsigned NOT NULL DEFAULT 0,
  `page_random` double unsigned NOT NULL DEFAULT 0,
  `page_touched` varbinary(14) NOT NULL DEFAULT '',
  `page_links_updated` varbinary(14) DEFAULT NULL,
  `page_latest` int(8) unsigned NOT NULL DEFAULT 0,
  `page_len` int(8) unsigned NOT NULL DEFAULT 0,
  `page_content_model` varbinary(32) DEFAULT NULL,
  `page_lang` varbinary(35) DEFAULT NULL,
  PRIMARY KEY (`page_id`),
  UNIQUE KEY `name_title` (`page_namespace`,`page_title`),
  KEY `page_random` (`page_random`),
  KEY `page_len` (`page_len`),
  KEY `page_redirect_namespace_len` (`page_is_redirect`,`page_namespace`,`page_len`)
) ENGINE=InnoDB AUTO_INCREMENT=4104286 DEFAULT CHARSET=binary
ruwiki
*************************** 1. row ***************************
       Table: page
Create Table: CREATE TABLE `page` (
  `page_id` int(8) unsigned NOT NULL AUTO_INCREMENT,
  `page_namespace` int(11) NOT NULL DEFAULT 0,
  `page_title` varbinary(255) NOT NULL DEFAULT '',
  `page_restrictions` varbinary(255) DEFAULT '',
  `page_is_redirect` tinyint(1) unsigned NOT NULL DEFAULT 0,
  `page_is_new` tinyint(1) unsigned NOT NULL DEFAULT 0,
  `page_random` double unsigned NOT NULL DEFAULT 0,
  `page_touched` varbinary(14) NOT NULL DEFAULT '',
  `page_links_updated` varbinary(14) DEFAULT NULL,
  `page_latest` int(8) unsigned NOT NULL DEFAULT 0,
  `page_len` int(8) unsigned NOT NULL DEFAULT 0,
  `page_content_model` varbinary(32) DEFAULT NULL,
  `page_lang` varbinary(35) DEFAULT NULL,
  PRIMARY KEY (`page_id`),
  UNIQUE KEY `name_title` (`page_namespace`,`page_title`),
  KEY `page_random` (`page_random`),
  KEY `page_len` (`page_len`),
  KEY `page_redirect_namespace_len` (`page_is_redirect`,`page_namespace`,`page_len`)
) ENGINE=InnoDB AUTO_INCREMENT=8253303 DEFAULT CHARSET=binary

So for those with varbinary we can just:

alter table page change page_restrictions page_restrictions varbinary(255) DEFAULT ''

I may be confused, but doesn't that also remove the NOT NULL? Which is fine from the MediaWiki standpoint, but I'm a little surprised that's inexpensive enough. Is the original therefore also inexpensive enough on the wikis that already have tinyblob?

Would it help to append , ALGORITHM=INPLACE, LOCK=NONE to the alters? I think that will make it error out if it would have to acquire locks, e.g. if the alter with varbinary is executed on a host with tinyblob or vice versa.

So for those with varbinary we can just:

alter table page change page_restrictions page_restrictions varbinary(255) DEFAULT ''

I may be confused, but doesn't that also remove the NOT NULL? Which is fine from the MediaWiki standpoint, but I'm a little surprised that's inexpensive enough. Is the original therefore also inexpensive enough on the wikis that already have tinyblob?

Yeah, if they already have tinyblob that's the same and either the original one or the DEFAULT ' ' won't lock the table. The problem is that on this particular wikis (s6) they all have varbinary
They are both expensive - I am not too worried about how long it takes, but how much it locks the table, for the masters. The slaves will be depooled anyways.

So to be clear, for those having tinyblob, I will apply the original patch. For the varbinary ones, I will probably also apply the normal patch on the slaves, but on the masters I will just change it to DEFAULT ' ' which doesn't lock the table so we avoid the master failovers.
If we wanted to, we could wait for the DC switchover, as that's maybe happening by the end of April - but with all this global pandemic situation, that's not guaranteed.

Would it help to append , ALGORITHM=INPLACE, LOCK=NONE to the alters? I think that will make it error out if it would have to acquire locks, e.g. if the alter with varbinary is executed on a host with tinyblob or vice versa.

No, it won't. MySQL tries to use the less locking possible and always defaults to INPLACE if possible. Even if you specify it, it will fail as soon as it realises you are doing a datatype change as you correctly stated.

Marostegui updated the task description. (Show Details)Mar 25 2020, 6:31 AM
Marostegui updated the task description. (Show Details)Mar 25 2020, 7:36 AM
Marostegui updated the task description. (Show Details)Mar 26 2020, 8:29 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-26T08:44:41Z] <marostegui> Deploy schema change on s5 codfw, lag will show up on codfw - T248333

Marostegui updated the task description. (Show Details)Mar 26 2020, 9:21 AM
Marostegui added a comment.EditedMar 26 2020, 9:50 AM
  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1003
  • db1130
  • db1124
  • db1113
  • db1110
  • db1102
  • db1100
  • db1097
  • db1096
  • db1082
Marostegui updated the task description. (Show Details)Mar 27 2020, 7:52 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-27T07:58:56Z] <marostegui> Deploy schema change on s2 codfw - this will generate lag on s2 codfw - T248333

To make things even more interesting with the drift between tinyblob and varbinary we have seen that 10.4 now (from 10.2 onwards https://mariadb.com/kb/en/blob/) allows DEFAULT values for BLOB.

So instead of applying the same DEFAULT ' ' on the tinyblob for 10.4, there, we'll apply DEFAULT NULL so a NULL is inserted instead of ' '.

The definition on 10.1 is:

`page_restrictions` tinyblob,

And that results on a NULL being correctly inserted on page_restrictions column:

[test]> insert into page (page_id) values (1);
Query OK, 1 row affected (0.03 sec)

[test]> select * from page;
+---------+----------------+------------+-------------------+------------------+-------------+-------------+--------------+--------------------+-------------+----------+--------------------+-----------+
| page_id | page_namespace | page_title | page_restrictions | page_is_redirect | page_is_new | page_random | page_touched | page_links_updated | page_latest | page_len | page_content_model | page_lang |
+---------+----------------+------------+-------------------+------------------+-------------+-------------+--------------+--------------------+-------------+----------+--------------------+-----------+
|       1 |              0 |            | NULL              |                0 |           0 |           0 |              | NULL               |           0 |        0 | NULL               | NULL      |
+---------+----------------+------------+-------------------+------------------+-------------+-------------+--------------+--------------------+-------------+----------+--------------------+-----------+
1 row in set (0.04 sec)

However on 10.4 that doesn't happen unless forced:

`page_restrictions` tinyblob DEFAULT '',,

This results on:

[test]> insert into page (page_id) values (1);
Query OK, 1 row affected (0.00 sec)

[test]> select * from page;
+---------+----------------+------------+-------------------+------------------+-------------+-------------+--------------+--------------------+-------------+----------+--------------------+-----------+
| page_id | page_namespace | page_title | page_restrictions | page_is_redirect | page_is_new | page_random | page_touched | page_links_updated | page_latest | page_len | page_content_model | page_lang |
+---------+----------------+------------+-------------------+------------------+-------------+-------------+--------------+--------------------+-------------+----------+--------------------+-----------+
|       1 |              0 |            |                   |                0 |           0 |           0 |              | NULL               |           0 |        0 | NULL               | NULL      |
+---------+----------------+------------+-------------------+------------------+-------------+-------------+--------------+--------------------+-------------+----------+--------------------+-----------+
1 row in set (0.00 sec)

Even though replication doesn't break, it is better to get a NULL there too, just to be consistent with 10.1 behaviour
So by running:

alter table page change column page_restrictions page_restrictions tinyblob default NULL;

We get the desired same behaviour on 10.4:

 insert into page (page_id,page_title) values (2,'test');
Query OK, 1 row affected (0.00 sec)

[test]> select * from page;
+---------+----------------+------------+-------------------+------------------+-------------+-------------+--------------+--------------------+-------------+----------+--------------------+-----------+
| page_id | page_namespace | page_title | page_restrictions | page_is_redirect | page_is_new | page_random | page_touched | page_links_updated | page_latest | page_len | page_content_model | page_lang |
+---------+----------------+------------+-------------------+------------------+-------------+-------------+--------------+--------------------+-------------+----------+--------------------+-----------+
|       1 |              0 |            |                   |                0 |           0 |           0 |              | NULL               |           0 |        0 | NULL               | NULL      |
|       2 |              0 | test       | NULL              |                0 |           0 |           0 |              | NULL               |           0 |        0 | NULL               | NULL      |
+---------+----------------+------------+-------------------+------------------+-------------+-------------+--------------+--------------------+-------------+----------+--------------------+-----------+

[test]> show create table page\G
*************************** 1. row ***************************
       Table: page
Create Table: CREATE TABLE `page` (
  `page_id` int(8) unsigned NOT NULL AUTO_INCREMENT,
  `page_namespace` int(11) NOT NULL DEFAULT 0,
  `page_title` varbinary(255) NOT NULL DEFAULT '',
  `page_restrictions` tinyblob DEFAULT NULL,
  `page_is_redirect` tinyint(1) unsigned NOT NULL DEFAULT 0,
  `page_is_new` tinyint(1) unsigned NOT NULL DEFAULT 0,
  `page_random` double unsigned NOT NULL DEFAULT 0,
  `page_touched` varbinary(14) NOT NULL DEFAULT '',
  `page_links_updated` varbinary(14) DEFAULT NULL,
  `page_latest` int(8) unsigned NOT NULL DEFAULT 0,
  `page_len` int(8) unsigned NOT NULL DEFAULT 0,
  `page_content_model` varbinary(32) DEFAULT NULL,
  `page_lang` varbinary(35) DEFAULT NULL,
  PRIMARY KEY (`page_id`),
  UNIQUE KEY `name_title` (`page_namespace`,`page_title`),
  KEY `page_random` (`page_random`),
  KEY `page_len` (`page_len`),
  KEY `page_redirect_namespace_len` (`page_is_redirect`,`page_namespace`,`page_len`)
) ENGINE=InnoDB AUTO_INCREMENT=5345116 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
1 row in set (0.00 sec)

We have 1 host per section in production running 10.4 (and Debian Buster) T246604. So the good thing is that replication doesn't break, but it might if we had a 10.4 master and a 10.1 slave, which is unlikely to happen as we always upgrade the master the last.

Mentioned in SAL (#wikimedia-operations) [2020-03-27T10:01:20Z] <marostegui> Alter db1096:3315 enwikivoyage.page to set page_restrictions to default NULL - T248333

Mentioned in SAL (#wikimedia-operations) [2020-03-27T10:02:01Z] <marostegui> Alter db2084:3315 enwikivoyage.page to set page_restrictions to default NULL - T248333

s5 hosts "fixed"

db1096:3315
*************************** 1. row ***************************
@@version: 10.4.12-MariaDB-log
*************************** 1. row ***************************
       Table: page
Create Table: CREATE TABLE `page` (
  `page_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `page_namespace` int(11) NOT NULL,
  `page_title` varbinary(255) NOT NULL,
  `page_restrictions` tinyblob DEFAULT NULL,
  `page_is_redirect` tinyint(3) unsigned NOT NULL DEFAULT 0,
  `page_is_new` tinyint(3) unsigned NOT NULL DEFAULT 0,
  `page_random` double unsigned NOT NULL,
  `page_touched` binary(14) NOT NULL DEFAULT '\0\0\0\0\0\0\0\0\0\0\0\0\0\0',
  `page_links_updated` varbinary(14) DEFAULT NULL,
  `page_latest` int(10) unsigned NOT NULL,
  `page_len` int(10) unsigned NOT NULL,
  `page_content_model` varbinary(32) DEFAULT NULL,
  `page_lang` varbinary(35) DEFAULT NULL,
  PRIMARY KEY (`page_id`),
  UNIQUE KEY `name_title` (`page_namespace`,`page_title`),
  KEY `page_random` (`page_random`),
  KEY `page_len` (`page_len`),
  KEY `page_redirect_namespace_len` (`page_is_redirect`,`page_namespace`,`page_len`)
) ENGINE=InnoDB AUTO_INCREMENT=177785 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED
db2084:3315
*************************** 1. row ***************************
@@version: 10.4.12-MariaDB-log
*************************** 1. row ***************************
       Table: page
Create Table: CREATE TABLE `page` (
  `page_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `page_namespace` int(11) NOT NULL,
  `page_title` varbinary(255) NOT NULL,
  `page_restrictions` tinyblob DEFAULT NULL,
  `page_is_redirect` tinyint(3) unsigned NOT NULL DEFAULT 0,
  `page_is_new` tinyint(3) unsigned NOT NULL DEFAULT 0,
  `page_random` double unsigned NOT NULL,
  `page_touched` binary(14) NOT NULL DEFAULT '\0\0\0\0\0\0\0\0\0\0\0\0\0\0',
  `page_links_updated` varbinary(14) DEFAULT NULL,
  `page_latest` int(10) unsigned NOT NULL,
  `page_len` int(10) unsigned NOT NULL,
  `page_content_model` varbinary(32) DEFAULT NULL,
  `page_lang` varbinary(35) DEFAULT NULL,
  PRIMARY KEY (`page_id`),
  UNIQUE KEY `name_title` (`page_namespace`,`page_title`),
  KEY `page_random` (`page_random`),
  KEY `page_len` (`page_len`),
  KEY `page_redirect_namespace_len` (`page_is_redirect`,`page_namespace`,`page_len`)
) ENGINE=InnoDB AUTO_INCREMENT=177785 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED

Mentioned in SAL (#wikimedia-operations) [2020-03-27T10:28:46Z] <marostegui> Alter db2125 s2 to set page_restrictions to default NULL - T248333

db2125 from s2 "fixed" on all its wikis

Marostegui updated the task description. (Show Details)Mar 27 2020, 10:29 AM
Marostegui added a comment.EditedMar 27 2020, 11:21 AM

s2 eqiad progress

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1004
  • db1140
  • db1129
  • db1125
  • db1122
  • db1105
  • db1103
  • db1090
  • db1076
  • db1074
Marostegui updated the task description. (Show Details)Mar 27 2020, 12:04 PM

Good to learn that tinyblob NOT NULL DEFAULT '' works correctly on 10.4, so in the future once everything is on 10.4 we can do that instead of having to do the more expensive change to tinyblob NULL.

Note that in terms of mediawiki in general (not WMF-specific)- MySQL hasn't adapted the default for blobs of any kind, so mw tables outside of WMF will still have to work (for other tables, in general) for the lowest common denominator.

Marostegui updated the task description. (Show Details)Mar 30 2020, 7:24 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-30T07:26:39Z] <marostegui> Deploy schema change on s4 codfw, this will generate lag on codfw - T248333

Mentioned in SAL (#wikimedia-operations) [2020-03-30T07:28:42Z] <marostegui> Deploy schema change on labswiki (wikitech) - T248333

Marostegui updated the task description. (Show Details)Mar 30 2020, 7:30 AM

@Andrew I have executed the alter on labswiki. Can you run it on labstestwiki? Thanks!

Marostegui updated the task description. (Show Details)Mar 30 2020, 11:30 AM
Marostegui added a comment.EditedMar 30 2020, 1:12 PM

s4 eqiad progress

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1004
  • db1138
  • db1125
  • db1121
  • db1103
  • db1102
  • db1097
  • db1091
  • db1084
  • db1081
mysql:root@localhost [labtestwiki]> alter table page change page_restrictions page_restrictions varbinary(255) DEFAULT '';
Query OK, 27971 rows affected (3.80 sec)               
Records: 27971  Duplicates: 0  Warnings: 0

Is that it?

Yes! Thank you :)

Marostegui updated the task description. (Show Details)Mar 31 2020, 5:09 AM
Marostegui updated the task description. (Show Details)Mar 31 2020, 7:33 AM
Marostegui updated the task description. (Show Details)Apr 1 2020, 9:32 AM
Marostegui updated the task description. (Show Details)Apr 1 2020, 10:46 AM
Marostegui added a comment.EditedApr 1 2020, 12:08 PM

s8 eqiad progress

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1005
  • db1126
  • db1124
  • db1116
  • db1111
  • db1109
  • db1104
  • db1101
  • db1099
  • db1092
  • db1087
Marostegui updated the task description. (Show Details)Apr 3 2020, 7:49 AM
Marostegui updated the task description. (Show Details)
Marostegui updated the task description. (Show Details)Apr 3 2020, 8:42 AM
Marostegui added a comment.EditedApr 3 2020, 9:18 AM

s7 eqiad

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1003
  • db1136
  • db1125
  • db1116
  • db1101
  • db1098
  • db1094
  • db1090
  • db1086
  • db1079
Marostegui updated the task description. (Show Details)Apr 6 2020, 6:16 AM
Marostegui updated the task description. (Show Details)
Marostegui updated the task description. (Show Details)Apr 6 2020, 7:19 AM
Marostegui added a comment.EditedApr 6 2020, 8:49 AM

s1 eqiad

  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1003
  • db1139
  • db1134
  • db1124
  • db1119
  • db1118
  • db1107
  • db1106
  • db1105
  • db1099
  • db1089
  • db1083
  • db1080

s3 is of course full of drifts.
From s3.dblist we have 126 wikis with tinyblob and 774 with varbinary

Marostegui updated the task description. (Show Details)Apr 7 2020, 10:31 AM
Marostegui updated the task description. (Show Details)Apr 8 2020, 5:32 AM
Marostegui added a comment.EditedApr 8 2020, 6:30 AM
  • labsdb1012
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1004
  • db1124
  • db1123
  • db1112
  • db1095
  • db1078
  • db1075
Marostegui closed this task as Resolved.Apr 9 2020, 2:17 PM
Marostegui updated the task description. (Show Details)

This is all done

Aklapper removed a subscriber: Anomie.Fri, Oct 16, 5:41 PM