Page MenuHomePhabricator

Schema change to make archive.ar_rev_id NOT NULL
Closed, ResolvedPublic

Description

This is a cleanup, there should be no harm in waiting until it can be combined with other changes to the archive table that require depooling replicas. I know of several coming up in the foreseeable future:

  • Dropping ar_text and ar_flags (https://gerrit.wikimedia.org/r/c/393929/, currently blocked on T185128)
  • A patch to drop ar_comment once the comment table migration is complete.
  • A patch to drop ar_user and ar_user_text once the actor table migration is complete.
  • A patch for T183487 to drop ar_text_id, ar_content_model, and ar_content_format once MCR migration is complete.

This must not be run until T191307 is resolved. But since I plan to take care of that either this week or next, while I fully expect this to not be started for months, that shouldn't be a problem.

From the checklist at https://wikitech.wikimedia.org/wiki/Schema_changes:

  1. ALTERs to run: https://gerrit.wikimedia.org/r/c/402932/8/maintenance/archives/patch-ar_rev_id-not-null.sql
  2. Where to run those changes: all.dblist
  3. When to run those changes: No time constraint. See details above.
  4. If the schema change is backwards compatible: Yes.
  5. If the schema change has been tested already on some of the test/beta wikis: It'll be automatically deployed to Beta Cluster soon enough, if it hasn't already.
  6. if the data should be made available on the labs replicas and/or dumps: N/A, there is no new data in this change

Schema change progress:

Details

Related Gerrit Patches:
operations/mediawiki-config : masterdb-eqiad.php: Depool db1077
operations/mediawiki-config : masterdb-eqiad.php: Depool db1123
operations/mediawiki-config : masterdb-eqiad.php: Depool db1078
operations/mediawiki-config : masterdb-eqiad.php: Depool db1101:3317
operations/puppet : productionjobqueue_redis: remove restarting cron in redis
operations/mediawiki-config : masterdb-eqiad.php: Depool db1098:3317
operations/mediawiki-config : masterdb-eqiad.php: Depool db1090:3317
operations/mediawiki-config : masterdb-eqiad.php: Depool db1079
operations/mediawiki-config : masterdb-eqiad.php: Depool db1094
operations/mediawiki-config : masterdb-eqiad.php: Depool db1086
operations/mediawiki-config : masterdb-eqiad.php: Depool db1087
operations/mediawiki-config : masterdb-eqiad.php: Depool db1109
operations/mediawiki-config : masterdb-eqiad.php: Depool db1092
operations/mediawiki-config : masterdb-eqiad.php: Depool db1104
operations/mediawiki-config : masterdb-eqiad.php: Depool db1101:3318
operations/mediawiki-config : masterdb-eqiad.php: Depool db1099:3318
operations/mediawiki-config : masterdb-eqiad.php: Depool db1076
operations/mediawiki-config : masterdb-eqiad.php: Depool db1074
operations/mediawiki-config : masterdb-eqiad.php: Depool db1090:3312
operations/mediawiki-config : masterdb-eqiad.php: Depool db1105:3312
operations/mediawiki-config : masterdb-eqiad.php: Depool db1103:3312
operations/mediawiki-config : masterdb-eqiad.php: Depool db1122
operations/mediawiki-config : masterdb-eqiad.php: Depool db1106
operations/mediawiki-config : masterdb-eqiad.php: Depool db1080
operations/mediawiki-config : masterdb-eqiad.php: Depool db1119
operations/mediawiki-config : masterdb-eqiad.php: Depool db1067
operations/mediawiki-config : masterdb-eqiad.php: Depool db1114
operations/mediawiki-config : masterdb-eqiad.php: Depool db1083
operations/mediawiki-config : masterdb-eqiad.php: Depool db1105:3311
operations/mediawiki-config : masterdb-eqiad.php: Depool db1089
operations/mediawiki-config : masterdb-eqiad.php: Depool db1099:3311
operations/mediawiki-config : masterdb-eqiad.php: Depool db1121
operations/mediawiki-config : masterdb-eqiad.php: Depool db1091
operations/mediawiki-config : masterdb-eqiad.php: Depool db1084
operations/mediawiki-config : masterdb-eqiad.php: Depool db1081
operations/mediawiki-config : masterdb-eqiad.php: Depool db1103:3314
operations/mediawiki-config : masterdb-eqiad.php: Depool db1097:3314
operations/mediawiki-config : masterdb-eqiad.php: Depool db1113:3316
operations/mediawiki-config : masterdb-eqiad.php: Depool db1088
operations/mediawiki-config : masterdb-eqiad.php: Depool db1085
operations/mediawiki-config : masterdb-eqiad.php: Depool db1093
operations/mediawiki-config : masterdb-eqiad.php: Depool db1098:3316
operations/mediawiki-config : masterdb-eqiad.php: Depool db1096:3316
operations/mediawiki-config : masterdb-eqiad.php: Depool db1110
operations/mediawiki-config : masterdb-eqiad.php: Depool db1100
operations/mediawiki-config : masterdb-eqiad.php: Depool db1097:3315
operations/mediawiki-config : masterdb-eqiad.php: Depool db1082
operations/mediawiki-config : masterdb-eqiad.php: Depool db1096:3315
operations/mediawiki-config : masterdb-eqiad.php: Depool db1113:3315
operations/mediawiki-config : masterdb-codfw.php: Depool db2066
operations/mediawiki-config : masterdb-codfw.php: Depool db2084:3315
operations/mediawiki-config : masterdb-codfw.php: Depool db2038
operations/mediawiki-config : masterdb-eqiad.php: Depool db2089:3315
operations/mediawiki-config : masterdb-codfw.php: Depool db2059

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 442086 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1105:3312

https://gerrit.wikimedia.org/r/442086

Change 442086 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1105:3312

https://gerrit.wikimedia.org/r/442086

Mentioned in SAL (#wikimedia-operations) [2018-06-26T12:15:04Z] <marostegui> Deploy schema change on db1105:3312 T191316 T192926 T89737 T195193

Change 442236 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1090:3312

https://gerrit.wikimedia.org/r/442236

Change 442236 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1090:3312

https://gerrit.wikimedia.org/r/442236

Mentioned in SAL (#wikimedia-operations) [2018-06-27T05:08:44Z] <marostegui> Deploy schema change on db1090:3312 T191316 T192926 T89737 T195193

Change 442247 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1074

https://gerrit.wikimedia.org/r/442247

Change 442247 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1074

https://gerrit.wikimedia.org/r/442247

Mentioned in SAL (#wikimedia-operations) [2018-06-27T08:35:26Z] <marostegui> Deploy schema change on db1074 with replication, this will generate lag on s2 on labsdb T191316 T192926 T89737 T195193

Change 442267 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1076

https://gerrit.wikimedia.org/r/442267

Change 442267 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1076

https://gerrit.wikimedia.org/r/442267

Mentioned in SAL (#wikimedia-operations) [2018-06-27T10:24:09Z] <marostegui> Deploy schema change on db1076 T191316 T192926 T89737 T195193

Mentioned in SAL (#wikimedia-operations) [2018-06-27T11:26:52Z] <marostegui> Deploy schema change on s8 codfw master (db2045) with replication, this will generate lag on s8 codfw T191316 T192926 T89737 T195193

Marostegui updated the task description. (Show Details)Jun 27 2018, 4:38 PM

Mentioned in SAL (#wikimedia-operations) [2018-06-27T16:39:23Z] <marostegui> Deploy schema change on dbstore1002:s8 T191316 T192926 T89737 T195193

Marostegui added a comment.EditedJun 28 2018, 4:37 AM

s8 eqiad progress

  • dbstore1002
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • db1109
  • db1104
  • db1124
  • db1101
  • db1099
  • db1092
  • db1087
  • db1071
Marostegui updated the task description. (Show Details)Jun 28 2018, 4:39 AM

Change 442758 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1099:3318

https://gerrit.wikimedia.org/r/442758

Change 442758 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1099:3318

https://gerrit.wikimedia.org/r/442758

Mentioned in SAL (#wikimedia-operations) [2018-06-28T04:46:19Z] <marostegui> Deploy schema change on db1099:3318 T191316 T192926 T89737 T195193

Change 442869 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1101:3318

https://gerrit.wikimedia.org/r/442869

Change 442869 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1101:3318

https://gerrit.wikimedia.org/r/442869

Mentioned in SAL (#wikimedia-operations) [2018-06-28T15:01:53Z] <marostegui> Deploy schema change on db1101:3318 T191316 T192926 T89737 T195193

Change 443022 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1104

https://gerrit.wikimedia.org/r/443022

Change 443022 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1104

https://gerrit.wikimedia.org/r/443022

Mentioned in SAL (#wikimedia-operations) [2018-06-29T05:02:02Z] <marostegui> Deploy schema change on db1104 T191316 T192926 T89737 T195193

Change 443067 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1092

https://gerrit.wikimedia.org/r/443067

Change 443067 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1092

https://gerrit.wikimedia.org/r/443067

Mentioned in SAL (#wikimedia-operations) [2018-06-29T10:20:01Z] <marostegui> Deploy schema change on db1092 T191316 T192926 T89737 T195193

Change 443345 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1087

https://gerrit.wikimedia.org/r/443345

Change 443345 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1109

https://gerrit.wikimedia.org/r/443345

Mentioned in SAL (#wikimedia-operations) [2018-07-02T04:59:10Z] <marostegui> Deploy schema change on db1109 T191316 T192926 T89737 T195193

Change 443379 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1092

https://gerrit.wikimedia.org/r/443379

Change 443379 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1087

https://gerrit.wikimedia.org/r/443379

Mentioned in SAL (#wikimedia-operations) [2018-07-02T09:24:00Z] <marostegui> Deploy schema change on db1087 with replication, this will generate lag on s8 on labs T191316 T192926 T89737 T195193

Mentioned in SAL (#wikimedia-operations) [2018-07-02T12:49:28Z] <marostegui> Deploy schema change on s7 codfw master (db2040) with replication, this will generate lag on s7 codfw T191316 T192926 T89737 T195193

Marostegui updated the task description. (Show Details)Jul 2 2018, 3:35 PM

Mentioned in SAL (#wikimedia-operations) [2018-07-02T15:38:26Z] <marostegui> Deploy schema change on dbstore1002:s7 T191316 T192926 T89737 T195193 T197459

Marostegui added a comment.EditedJul 2 2018, 3:41 PM

s7 eqiad progress

  • labsdb1009
  • labsdb1010
  • labsdb1011
  • db1125
  • dbstore1002
  • db1079
  • db1086
  • db1090
  • db1094
  • db1098
  • db1101
  • db1062
Marostegui updated the task description. (Show Details)Jul 2 2018, 3:42 PM

Mentioned in SAL (#wikimedia-operations) [2018-07-03T04:27:19Z] <marostegui> Deploy schema change on s2 primary master db1066 T191316 T192926 T195193

Change 443537 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1086

https://gerrit.wikimedia.org/r/443537

Change 443537 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1086

https://gerrit.wikimedia.org/r/443537

Mentioned in SAL (#wikimedia-operations) [2018-07-03T04:40:21Z] <marostegui> Deploy schema change on db1086 T191316 T192926 T89737 T195193 T197459

Marostegui updated the task description. (Show Details)Jul 3 2018, 5:04 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-03T05:04:44Z] <marostegui> Deploy schema change on s8 primary master db1071 T191316 T192926 T195193

Change 443544 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1094

https://gerrit.wikimedia.org/r/443544

Change 443544 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1094

https://gerrit.wikimedia.org/r/443544

Mentioned in SAL (#wikimedia-operations) [2018-07-03T05:26:33Z] <marostegui> Deploy schema change on db1094 T191316 T192926 T89737 T195193 T197459

Change 443547 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1079

https://gerrit.wikimedia.org/r/443547

Change 443547 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1079

https://gerrit.wikimedia.org/r/443547

Mentioned in SAL (#wikimedia-operations) [2018-07-03T06:14:24Z] <marostegui> Deploy schema change on db1079 with replication, this will generate lag on s7 on labsdb hosts T191316 T192926 T89737 T195193 T197459

Change 443554 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1090:3317

https://gerrit.wikimedia.org/r/443554

Change 443554 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1090:3317

https://gerrit.wikimedia.org/r/443554

Change 443555 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] jobqueue_redis: remove restarting cron in redis

https://gerrit.wikimedia.org/r/443555

Mentioned in SAL (#wikimedia-operations) [2018-07-03T07:13:34Z] <marostegui> Deploy schema change on db1090:3317 T191316 T192926 T89737 T195193 T197459

Marostegui updated the task description. (Show Details)Jul 3 2018, 7:25 AM

Change 443577 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1098:3317

https://gerrit.wikimedia.org/r/443577

Change 443577 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1098:3317

https://gerrit.wikimedia.org/r/443577

Mentioned in SAL (#wikimedia-operations) [2018-07-03T09:13:13Z] <marostegui> Deploy schema change on db1098:3317 T191316 T192926 T89737 T195193 T197459

s3 is always messy, the databases that already have this changed applied as they were created before the merge and thus need to be excluded from this alter:

bnwikivoyage
euwikisource
gorwiki
hiwikimedia
idwikimedia
inhwiki
lfnwiki
pmswikisource
pswikivoyage
romdwikimedia
sahwikiquote

Mentioned in SAL (#wikimedia-operations) [2018-07-03T09:58:59Z] <marostegui> Deploy schema change on s3 codfw primary master (db2043) with replication, this will generate lag on s3 codfw T191316 T192926 T89737 T195193 T197459

Marostegui updated the task description. (Show Details)Jul 3 2018, 12:54 PM

Mentioned in SAL (#wikimedia-operations) [2018-07-03T13:19:35Z] <marostegui> Deploy schema change on dbstore1002:s3 T191316 T192926 T89737 T195193

Marostegui added a comment.EditedJul 3 2018, 1:24 PM

s3 eqiad progress

  • labsdb1009
  • labsdb1010
  • labsdb1011
  • dbstore1002
  • db1124
  • db1077
  • db1078
  • db1123
  • db1075
Marostegui updated the task description. (Show Details)Jul 3 2018, 1:26 PM

Change 443555 merged by Giuseppe Lavagetto:
[operations/puppet@production] jobqueue_redis: remove restarting cron in redis

https://gerrit.wikimedia.org/r/443555

Change 443763 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1101:3317

https://gerrit.wikimedia.org/r/443763

Change 443763 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1101:3317

https://gerrit.wikimedia.org/r/443763

Mentioned in SAL (#wikimedia-operations) [2018-07-04T04:59:02Z] <marostegui> Deploy schema change on db1101:3317 T191316 T192926 T89737 T195193

Change 443772 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] redis: remove cronjob for restarts on slaves as well

https://gerrit.wikimedia.org/r/443772

Change 443773 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] redis: remove now-useless specific classes

https://gerrit.wikimedia.org/r/443773

Change 443782 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1078

https://gerrit.wikimedia.org/r/443782

Change 443782 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1078

https://gerrit.wikimedia.org/r/443782

Mentioned in SAL (#wikimedia-operations) [2018-07-04T08:00:16Z] <marostegui> Deploy schema change on db1078 T191316 T192926 T89737 T195193

Change 443791 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1123

https://gerrit.wikimedia.org/r/443791

Change 443791 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1123

https://gerrit.wikimedia.org/r/443791

Mentioned in SAL (#wikimedia-operations) [2018-07-04T08:54:03Z] <marostegui> Deploy schema change on db1123 T191316 T192926 T89737 T195193

Change 443815 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1077

https://gerrit.wikimedia.org/r/443815

Change 443815 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1077

https://gerrit.wikimedia.org/r/443815

Mentioned in SAL (#wikimedia-operations) [2018-07-04T10:46:34Z] <marostegui> Deploy schema change on db1077 with replication, this will generate lag on labs s3 T191316 T192926 T89737 T195193

Mentioned in SAL (#wikimedia-operations) [2018-07-05T04:42:04Z] <marostegui> Deploy schema change on s7 primary master (db1062) T191316 T192926 T195193

Marostegui updated the task description. (Show Details)Jul 5 2018, 5:28 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-05T05:29:07Z] <marostegui> Deploy schema change on s3 primary master (db1075) T191316 T192926 T195193

Marostegui updated the task description. (Show Details)Jul 5 2018, 6:03 AM

This is all done.
I am not closing this task because s1 master is still pending and will be failed over the 18th July (T197069) to a host that already have the schema change, once that is failed over, I will close this task.

Marostegui closed this task as Resolved.Jul 18 2018, 8:34 AM
Marostegui updated the task description. (Show Details)

db1052 was failed over today and now got the schema change so this is all done.

Mentioned in SAL (#wikimedia-operations) [2018-07-23T08:11:54Z] <marostegui> Apply schema change  T191316 to labswiki and labstestwiki T200140