Page MenuHomePhabricator

Fix mismatching field type of user table for columns user_email_authenticated, user_email_token, user_email_token_expires, user_newpass_time, user_registration, user_token, user_touched on wmf wikis
Closed, ResolvedPublic

Description

https://drift-tracker.toolforge.org/report/core/ shows a schema drift for:

  • Mismatching field type of user.user_email_authenticated
  • Mismatching field type of user.user_email_token
  • Mismatching field type of user.user_email_token_expires
  • Mismatching field type of user.user_newpass_time
  • Mismatching field type of user.user_touched
  • Mismatching field type of user.user_token
  • Mismatching field type of user.user_registration

on s1, s2, s3, s4, s7

  • Mismatching field nullability of user.user_email
  • Mismatching field type of user.user_newpassword
  • Mismatching field nullability of user.user_newpassword
  • Mismatching field type of user.user_password
  • Mismatching field nullability of user.user_password

on s3, s7

  • Mismatching field type of user.user_email

on s3, s6, s7

production
root@db1112.eqiad.wmnet[thwikiquote]> show create table user\G
*************************** 1. row ***************************
       Table: user
Create Table: CREATE TABLE `user` (
  `user_id` int(5) unsigned NOT NULL AUTO_INCREMENT,
  `user_name` varbinary(255) NOT NULL DEFAULT '',
  `user_real_name` varbinary(255) NOT NULL DEFAULT '',
  `user_password` varbinary(255) DEFAULT NULL,
  `user_newpassword` varbinary(255) DEFAULT NULL,
  `user_email` varbinary(255) DEFAULT NULL,
  `user_touched` varbinary(14) NOT NULL,
  `user_token` varbinary(32) NOT NULL DEFAULT '',
  `user_email_authenticated` varbinary(14) DEFAULT NULL,
  `user_email_token` varbinary(32) DEFAULT NULL,
  `user_email_token_expires` varbinary(14) DEFAULT NULL,
  `user_registration` varbinary(14) DEFAULT NULL,
  `user_newpass_time` varbinary(14) DEFAULT NULL,
  `user_editcount` int(11) DEFAULT NULL,
  `user_password_expires` varbinary(14) DEFAULT NULL,
  PRIMARY KEY (`user_id`),
  UNIQUE KEY `user_name` (`user_name`),
  KEY `user_email_token` (`user_email_token`),
  KEY `user_email` (`user_email`(50))
) ENGINE=InnoDB AUTO_INCREMENT=5135 DEFAULT CHARSET=binary
1 row in set (0.001 sec)

The code has the following definition for the three columns:


code
user_email_authenticated BINARY(14) DEFAULT NULL,
user_email_token BINARY(32) DEFAULT NULL,
user_email_token_expires BINARY(14) DEFAULT NULL,

Addition in 77c833c4e69c01656e35ac940aa1d71b2a3690ba
Then char() binary rewrite to binary() in f38344aba87edbdb5dba23d6433bf3c7cbbb0b51
DEFAULT NULL added in 88bfadf5eddfb80b9cf189fa71e79bb16adff847

But production is using varbinary instead of binary (T42626)


code
user_newpass_time BINARY(14) DEFAULT NULL,

Addition in e9de4983bfbf4ad8cd13444633731e32997fa169#change-pVGjpnYAcwz0
Then char() binary rewrite to binary() in f38344aba87edbdb5dba23d6433bf3c7cbbb0b51
DEFAULT NULL added in 88bfadf5eddfb80b9cf189fa71e79bb16adff847

But production is using varbinary instead of binary (T42626)


code
user_touched BINARY(14) NOT NULL,

Addition in 77c833c4e69c01656e35ac940aa1d71b2a3690ba
Then char() binary rewrite to binary() in f38344aba87edbdb5dba23d6433bf3c7cbbb0b51
Default removed in c697406d3f67fa9cd90fad14cdd4dbf680daa4d8

But production is using varbinary instead of binary (T42626)


code
user_token BINARY(32) DEFAULT '' NOT NULL,

Addition in 72e66de074fe8be9250320012ce4d9abd5850471
Then char() binary rewrite to binary() in f38344aba87edbdb5dba23d6433bf3c7cbbb0b51

But production is using varbinary instead of binary


code
user_registration BINARY(14) DEFAULT NULL,

Addition in a35fcb0bed5f8600b4330039b6772fa4c7a49cd0
Then char() binary rewrite to binary() in f38344aba87edbdb5dba23d6433bf3c7cbbb0b51
Default removed in c697406d3f67fa9cd90fad14cdd4dbf680daa4d8

But production is using varbinary instead of binary (T42626)


code
CREATE TABLE /*_*/user (
...
  user_password TINYBLOB NOT NULL,
  user_newpassword TINYBLOB NOT NULL,
  user_email TINYTEXT NOT NULL,

Addition in d82c14fb4fbac288b42ca5918b0a72f33ecb1e69#change-pVGjpnYAcwz0
Default removed in 241fadbc29c95795cf7c7bb37ddd55539a1cc412

But production is using varbinary NULL instead of tinyblob NOT NULL


Please check production definition and adjust the columns

If the definition of production are better for big wikis, feel free to abandon and request code change with a new task (This task could be reused to apply the newest schema)

There is no ready ALTER statement in the commit history.

alter1
ALTER TABLE /*_*/user
  CHANGE user_email_authenticated user_email_authenticated BINARY(14) DEFAULT NULL,
  CHANGE user_email_token user_email_token BINARY(32) DEFAULT NULL,
  CHANGE user_email_token_expires user_email_token_expires BINARY(14) DEFAULT NULL,
  CHANGE user_newpass_time user_newpass_time BINARY(14) DEFAULT NULL,
  CHANGE user_touched user_touched BINARY(14) NOT NULL,
  CHANGE user_token user_token BINARY(32) DEFAULT '' NOT NULL,
  CHANGE user_registration user_registration BINARY(14) DEFAULT NULL;
  1. ALTERs to run: see above
  2. Where to run those changes: s1, s2, s4 - list of wikis needs to be created
  3. When to run those changes: any time
  4. If the schema change is backwards compatible: Yes
  5. If the schema change has been tested already on some of the test/beta wikis: beta cluster is running with the new schema
  6. if the data should be made available on the labs replicas and/or dumps: no change of the existing rules

Progress

  • s1
  • s2
  • s4
alter2
ALTER TABLE /*_*/user
  CHANGE user_email_authenticated user_email_authenticated BINARY(14) DEFAULT NULL,
  CHANGE user_email_token user_email_token BINARY(32) DEFAULT NULL,
  CHANGE user_email_token_expires user_email_token_expires BINARY(14) DEFAULT NULL,
  CHANGE user_newpass_time user_newpass_time BINARY(14) DEFAULT NULL,
  CHANGE user_touched user_touched BINARY(14) NOT NULL,
  CHANGE user_token user_token BINARY(32) DEFAULT '' NOT NULL,
  CHANGE user_registration user_registration BINARY(14) DEFAULT NULL,

  CHANGE user_password user_password TINYBLOB NOT NULL,
  CHANGE user_newpassword user_newpassword TINYBLOB NOT NULL,
  CHANGE user_email user_email TINYTEXT NOT NULL;
  1. ALTERs to run: see above
  2. Where to run those changes: s3, s7 - list of wikis needs to be created
  3. When to run those changes: any time
  4. If the schema change is backwards compatible: Yes
  5. If the schema change has been tested already on some of the test/beta wikis: beta cluster is running with the new schema
  6. if the data should be made available on the labs replicas and/or dumps: no change of the existing rules

Progress

  • s3
  • s7
alter3
ALTER TABLE /*_*/user
  CHANGE user_email user_email TINYTEXT NOT NULL;
  1. ALTERs to run: see above
  2. Where to run those changes: s6 - list of wikis needs to be created
  3. When to run those changes: any time
  4. If the schema change is backwards compatible: Yes
  5. If the schema change has been tested already on some of the test/beta wikis: beta cluster is running with the new schema
  6. if the data should be made available on the labs replicas and/or dumps: no change of the existing rules

Progress

  • s6

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Mentioned in SAL (#wikimedia-operations) [2022-05-01T23:12:27Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1157 (T298565)', diff saved to https://phabricator.wikimedia.org/P27187 and previous config saved to /var/cache/conftool/dbconfig/20220501-231227-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-01T23:57:42Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1157 (T298565)', diff saved to https://phabricator.wikimedia.org/P27195 and previous config saved to /var/cache/conftool/dbconfig/20220501-235742-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T00:14:51Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1157 (T298565)', diff saved to https://phabricator.wikimedia.org/P27197 and previous config saved to /var/cache/conftool/dbconfig/20220502-001449-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T00:30:52Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1157 (T298565)', diff saved to https://phabricator.wikimedia.org/P27198 and previous config saved to /var/cache/conftool/dbconfig/20220502-003052-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T01:16:08Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1157 (T298565)', diff saved to https://phabricator.wikimedia.org/P27207 and previous config saved to /var/cache/conftool/dbconfig/20220502-011607-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T01:33:17Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1157 (T298565)', diff saved to https://phabricator.wikimedia.org/P27210 and previous config saved to /var/cache/conftool/dbconfig/20220502-013316-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T01:50:28Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1157 (T298565)', diff saved to https://phabricator.wikimedia.org/P27213 and previous config saved to /var/cache/conftool/dbconfig/20220502-015028-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T02:35:43Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1157 (T298565)', diff saved to https://phabricator.wikimedia.org/P27217 and previous config saved to /var/cache/conftool/dbconfig/20220502-023543-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T02:35:57Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1112 (T298565)', diff saved to https://phabricator.wikimedia.org/P27218 and previous config saved to /var/cache/conftool/dbconfig/20220502-023556-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T03:55:15Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112 (T298565)', diff saved to https://phabricator.wikimedia.org/P27237 and previous config saved to /var/cache/conftool/dbconfig/20220502-035514-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T03:55:23Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P27238 and previous config saved to /var/cache/conftool/dbconfig/20220502-035522-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T04:11:42Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P27245 and previous config saved to /var/cache/conftool/dbconfig/20220502-041141-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T04:56:57Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P27254 and previous config saved to /var/cache/conftool/dbconfig/20220502-045656-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T05:14:02Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P27258 and previous config saved to /var/cache/conftool/dbconfig/20220502-051402-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T05:36:16Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P27263 and previous config saved to /var/cache/conftool/dbconfig/20220502-053615-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T06:21:31Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P27274 and previous config saved to /var/cache/conftool/dbconfig/20220502-062131-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T06:38:38Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P27282 and previous config saved to /var/cache/conftool/dbconfig/20220502-063837-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T06:54:42Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P27286 and previous config saved to /var/cache/conftool/dbconfig/20220502-065442-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T07:39:59Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P27302 and previous config saved to /var/cache/conftool/dbconfig/20220502-073958-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T07:40:07Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1179 (T298565)', diff saved to https://phabricator.wikimedia.org/P27303 and previous config saved to /var/cache/conftool/dbconfig/20220502-074006-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T07:56:45Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179 (T298565)', diff saved to https://phabricator.wikimedia.org/P27307 and previous config saved to /var/cache/conftool/dbconfig/20220502-075644-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-02T08:42:00Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179 (T298565)', diff saved to https://phabricator.wikimedia.org/P27320 and previous config saved to /var/cache/conftool/dbconfig/20220502-084200-ladsgroup.json

Ladsgroup renamed this task from Fix mismatching field type of user table for columns user_email_authenticated, user_email_token, user_email_token_expires, user_newpass_time, user_registration, user_token, user_touched, user_newpassword, user_password, user_email on wmf wikis to Fix mismatching field type of user table for columns user_email_authenticated, user_email_token, user_email_token_expires, user_newpass_time, user_registration, user_token, user_touched on wmf wikis.May 3 2022, 9:42 PM

This ticket has been already exploding and if you ask me, it should have been split to four or five tickets. For all drifts, this has been fixed except on masters of s1, s4, s5, s6 (db1118, db1160, db1173) and in some cases master of s7 (db1181). For the next round of clean up, I will do it in T307525: Fix mismatching field type of user table for columns user_newpassword, user_password, user_email on wmf wikis

Mentioned in SAL (#wikimedia-operations) [2022-05-20T13:45:18Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1118 (T298565)', diff saved to https://phabricator.wikimedia.org/P28183 and previous config saved to /var/cache/conftool/dbconfig/20220520-134515-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-05-20T14:41:13Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1118 (T298565)', diff saved to https://phabricator.wikimedia.org/P28192 and previous config saved to /var/cache/conftool/dbconfig/20220520-144111-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T09:22:59Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30345 and previous config saved to /var/cache/conftool/dbconfig/20220627-092256-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T09:27:11Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30346 and previous config saved to /var/cache/conftool/dbconfig/20220627-092710-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T10:12:26Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30364 and previous config saved to /var/cache/conftool/dbconfig/20220627-101226-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T10:12:39Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30365 and previous config saved to /var/cache/conftool/dbconfig/20220627-101235-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T10:16:43Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30368 and previous config saved to /var/cache/conftool/dbconfig/20220627-101643-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T11:01:58Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30385 and previous config saved to /var/cache/conftool/dbconfig/20220627-110158-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T11:02:09Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30386 and previous config saved to /var/cache/conftool/dbconfig/20220627-110207-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T11:06:25Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30390 and previous config saved to /var/cache/conftool/dbconfig/20220627-110624-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T11:51:40Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30399 and previous config saved to /var/cache/conftool/dbconfig/20220627-115140-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T11:51:53Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30400 and previous config saved to /var/cache/conftool/dbconfig/20220627-115148-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T11:56:05Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30401 and previous config saved to /var/cache/conftool/dbconfig/20220627-115604-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T12:41:24Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30419 and previous config saved to /var/cache/conftool/dbconfig/20220627-124124-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T12:41:37Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30420 and previous config saved to /var/cache/conftool/dbconfig/20220627-124132-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T12:46:49Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30421 and previous config saved to /var/cache/conftool/dbconfig/20220627-124648-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T13:32:04Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30439 and previous config saved to /var/cache/conftool/dbconfig/20220627-133204-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T13:32:16Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30440 and previous config saved to /var/cache/conftool/dbconfig/20220627-133212-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T13:36:27Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30441 and previous config saved to /var/cache/conftool/dbconfig/20220627-133627-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T14:21:42Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30445 and previous config saved to /var/cache/conftool/dbconfig/20220627-142142-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T14:21:55Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30446 and previous config saved to /var/cache/conftool/dbconfig/20220627-142151-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T14:26:08Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30449 and previous config saved to /var/cache/conftool/dbconfig/20220627-142607-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2022-06-27T15:11:23Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30458 and previous config saved to /var/cache/conftool/dbconfig/20220627-151123-ladsgroup.json

Ladsgroup moved this task from Blocked to Done on the DBA board.

With being done on old master of s3, this is officially done.