Page MenuHomePhabricator
Create Task
Maniphest T276156

Drop default of rc_timestamp
Closed, ResolvedPublic
Actions

Description

Part of T42626: Standardise type of timestamp database fields (MySQL) and T230428: Migrate tables.sql to abstract schema

  1. ALTERs to run: ALTER TABLE recentchanges ALTER rc_timestamp DROP DEFAULT;
  2. Where to run those changes: all.dblist
  3. When to run those changes: At any time
  4. If the schema change is backwards compatible: Yes
  5. If the schema change has been tested already on some of the test/beta wikis: Tested in beta cluster.
  6. if the data should be made available on the labs replicas and/or dumps: Yes, data in this table is public.

Progress

Related Objects

Event Timeline

Ladsgroup created this task.Mar 1 2021, 7:53 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 1 2021, 7:53 PM
• Marostegui claimed this task.Mar 1 2021, 7:54 PM
• Marostegui triaged this task as Medium priority.
• Marostegui added a project: DBA.
• Marostegui moved this task from Triage to Ready on the DBA board.
• Marostegui updated the task description. (Show Details)Mar 2 2021, 5:28 AM
• Marostegui added a comment.EditedMar 3 2021, 10:55 AM

I have altered db2089:3316 and will leave it running for a few days before going for a host in eqiad.
@Ladsgroup does this look good?

# mysql.py -hdb2089:3316 frwiki -e "show create table recentchanges\G" | grep -v KEY
*************************** 1. row ***************************
       Table: recentchanges
Create Table: CREATE TABLE `recentchanges` (
  `rc_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `rc_timestamp` varbinary(14) NOT NULL,
  `rc_actor` bigint(20) unsigned NOT NULL,
  `rc_namespace` int(11) NOT NULL DEFAULT 0,
  `rc_title` varbinary(255) NOT NULL DEFAULT '',
  `rc_comment_id` bigint(20) unsigned NOT NULL,
  `rc_minor` tinyint(3) unsigned NOT NULL DEFAULT 0,
  `rc_bot` tinyint(3) unsigned NOT NULL DEFAULT 0,
  `rc_new` tinyint(3) unsigned NOT NULL DEFAULT 0,
  `rc_cur_id` int(10) unsigned NOT NULL DEFAULT 0,
  `rc_this_oldid` int(10) unsigned NOT NULL DEFAULT 0,
  `rc_last_oldid` int(10) unsigned NOT NULL DEFAULT 0,
  `rc_type` tinyint(3) unsigned NOT NULL DEFAULT 0,
  `rc_source` varbinary(16) NOT NULL DEFAULT '',
  `rc_patrolled` tinyint(3) unsigned NOT NULL DEFAULT 0,
  `rc_ip` varbinary(40) NOT NULL DEFAULT '',
  `rc_old_len` int(10) DEFAULT NULL,
  `rc_new_len` int(10) DEFAULT NULL,
  `rc_deleted` tinyint(1) unsigned NOT NULL DEFAULT 0,
  `rc_logid` int(10) unsigned NOT NULL DEFAULT 0,
  `rc_log_type` varbinary(255) DEFAULT NULL,
  `rc_log_action` varbinary(255) DEFAULT NULL,
  `rc_params` blob DEFAULT NULL,
  • labsdb1011
  • labsdb1010
  • labsdb1009 (host has crashed T276980 - should get it if replication is ever back)
  • dbstore1005
  • db2141
  • db2129
  • db2124
  • db2117
  • db2114
  • db2097
  • db2095
  • db2089
  • db2087
  • db2076
  • db1173
  • db1168
  • db1155
  • db1140
  • db1139
  • db1131
  • db1125
  • db1113
  • db1098
  • db1096
  • db1088
  • db1085
  • clouddb1019
  • clouddb1015
  • clouddb1021
• Marostegui moved this task from Ready to In progress on the DBA board.Mar 3 2021, 10:55 AM
• Marostegui moved this task from Backlog to In progress on the Schema-change-in-production board.
• Marostegui added a comment.Mar 4 2021, 6:29 AM

@Ladsgroup this looks good too?

Ladsgroup added a comment.Mar 4 2021, 2:26 PM

oh I made a mistake. This needs to become binary(14) too. I added it to the wrong ticket (https://phabricator.wikimedia.org/transactions/detail/PHID-XACT-TASK-7r43urruzwsv4fj/) image table also might need it too but that's a separate issue.

• Marostegui added a comment.Mar 4 2021, 2:34 PM

@Ladsgroup let's include on this ticket then?
That makes this task also need a master swap for getting this change applied to the primary masters as changing the datatype isn't something that can be done online.

Ladsgroup added a comment.Mar 4 2021, 2:52 PM

This can be grouped with another schema change instead. Maybe with unsigned for rc_id? It doesn't need to happen at the same time as this change.

• Marostegui added a comment.Mar 4 2021, 2:53 PM

I am fine with that too, yes. Can you edit the other task then?

Ladsgroup added a comment.Mar 4 2021, 2:56 PM

Done

• Marostegui added a comment.Mar 4 2021, 2:58 PM

Thanks - I will add that change to the altered host and update that other task (T276150)

DannyS712 added a subscriber: DannyS712.Mar 7 2021, 11:51 PM

The removal of the rc_timestamp default has caused some extension tests to fail (T276734), it might make sense to wait to deploy the schema change until that is resolved to make sure nothing will break

Ladsgroup added a comment.Mar 8 2021, 3:01 AM

The tests failure doesn't have anything to do with this work. They inject fake rc entries to then query later with ores filtering (I wrote that extension and those faulty tests) and the tests need fixing. Nothing in production should inject an rc record without timestamp as it won't get purged after 30 days.

• Marostegui added a comment.Mar 8 2021, 5:56 AM

So is this ok to keep proceeding?

Ladsgroup added a comment.Mar 8 2021, 1:15 PM

Yup

Stashbot mentioned this in T276150: Schema change to make rc_id unsigned and rc_timestamp BINARY.Mar 9 2021, 6:19 AM

Mentioned in SAL (#wikimedia-operations) [2021-03-09T06:19:02Z] <marostegui> Deploy schema change on s6 codfw (there will be lag on codfw) T276150 T276156

• Marostegui updated the task description. (Show Details)Mar 9 2021, 6:49 AM
• Marostegui added a comment.Mar 9 2021, 9:48 AM

After s6 I will go for s7 as we need to switch that master in two weeks, we might as well get this schema change fully done on the master too

• Marostegui updated the task description. (Show Details)Mar 10 2021, 7:47 AM
Stashbot added a comment.Mar 10 2021, 7:52 AM

Mentioned in SAL (#wikimedia-operations) [2021-03-10T07:52:39Z] <marostegui> Deploy schema change on s7 codfw (lag will appear) T276150 T276156

• Marostegui updated the task description. (Show Details)Mar 10 2021, 8:39 AM
• Marostegui added a comment.EditedMar 10 2021, 9:43 AM

s7 progress

  • labsdb1011
  • labsdb1010
  • labsdb1009 (host has crashed T276980 - should get it if replication is ever back)
  • dbstore1003
  • db1174
  • db1170
  • db1155
  • db1136
  • db1127
  • db1125
  • db1116
  • db1101
  • db1098
  • db1086
  • db1079
  • clouddb1021
  • clouddb1018
  • clouddb1014
• Marostegui added a parent task: T274336: Switchover s7 from db1086 to db1136.Mar 11 2021, 7:29 AM
• Marostegui removed a parent task: T274336: Switchover s7 from db1086 to db1136.Mar 11 2021, 9:25 AM
• Marostegui updated the task description. (Show Details)
• Marostegui updated the task description. (Show Details)
• Marostegui updated the task description. (Show Details)Mar 11 2021, 9:29 AM
Stashbot added a comment.Mar 11 2021, 9:45 AM

Mentioned in SAL (#wikimedia-operations) [2021-03-11T09:45:30Z] <marostegui> Deploy schema change on s5 codfw master, lag will appear - T276150 T276156

• Marostegui updated the task description. (Show Details)Mar 11 2021, 9:54 AM
• Marostegui added a comment.EditedMar 11 2021, 10:05 AM

s5 progress

  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1003
  • db1154
  • db1150
  • db1145
  • db1144
  • db1130
  • db1124
  • db1113
  • db1110
  • db1100
  • db1096
  • db1082
  • clouddb1021
  • clouddb1020
  • clouddb1016
• Marostegui updated the task description. (Show Details)Mar 11 2021, 2:08 PM
• Marostegui updated the task description. (Show Details)Mar 12 2021, 6:28 AM
Stashbot added a comment.Mar 12 2021, 6:30 AM

Mentioned in SAL (#wikimedia-operations) [2021-03-12T06:30:43Z] <marostegui> Deploy schema change on s2 codfw master, lag will appear - T276150 T276156

• Marostegui updated the task description. (Show Details)Mar 12 2021, 7:03 AM
• Marostegui added a comment.EditedMar 12 2021, 7:30 AM

s2 eqiad progress

  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1004
  • db1171
  • db1170
  • db1162 (broken - will get it once it is reimaged)
  • db1155
  • db1146
  • db1129
  • db1125
  • db1122
  • db1105
  • db1076
  • db1074
  • clouddb1021
  • clouddb1018
  • clouddb1014
• Marostegui updated the task description. (Show Details)Mar 15 2021, 3:19 PM
• Marostegui updated the task description. (Show Details)Mar 16 2021, 6:02 AM
• Marostegui added a comment.EditedMar 16 2021, 8:21 AM

s8 progress

  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1005
  • db1172
  • db1154
  • db1126
  • db1124
  • db1116
  • db1114
  • db1111
  • db1109
  • db1104
  • db1101
  • db1099
  • db1087
  • clouddb1021
  • clouddb1020
  • clouddb1016
• Marostegui updated the task description. (Show Details)Mar 16 2021, 8:22 AM
Stashbot added a comment.Mar 17 2021, 11:49 AM

Mentioned in SAL (#wikimedia-operations) [2021-03-17T11:49:12Z] <marostegui> Deploy schema change on s8, lag will appear on wiki replicas T276150 T276156

• Marostegui updated the task description. (Show Details)Mar 18 2021, 7:18 AM
Stashbot added a comment.Mar 18 2021, 7:19 AM

Mentioned in SAL (#wikimedia-operations) [2021-03-18T07:19:32Z] <marostegui> Deploy schema change on s4 codfw master, lag will appear - T276150 T276156

• Marostegui updated the task description. (Show Details)Mar 18 2021, 9:23 AM
• Marostegui added a comment.EditedMar 18 2021, 9:38 AM
  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1004
  • db1160
  • db1155
  • db1150
  • db1149
  • db1148
  • db1147
  • db1146
  • db1145
  • db1144
  • db1143
  • db1142
  • db1141
  • db1138
  • db1125
  • db1121
  • clouddb1021
  • clouddb1019
  • clouddb1015
• Marostegui updated the task description. (Show Details)Mar 18 2021, 9:44 AM
• Marostegui updated the task description. (Show Details)Apr 5 2021, 9:49 AM
Stashbot added a comment.Apr 5 2021, 9:50 AM

Mentioned in SAL (#wikimedia-operations) [2021-04-05T09:50:36Z] <marostegui> Deploy schema change on s1 codfw, lag will appear in codfw - T276150 T276156

• Marostegui updated the task description. (Show Details)EditedApr 5 2021, 10:30 AM

s1 eqiad

  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1003
  • db1169
  • db1184
  • db1164
  • db1163
  • db1154
  • db1140
  • db1139
  • db1135
  • db1134
  • db1133
  • db1124
  • db1119
  • db1118
  • db1106
  • db1105
  • db1099
  • db1083
  • clouddb1021
  • clouddb1017
  • clouddb1013
• Marostegui updated the task description. (Show Details)Apr 7 2021, 11:38 AM
Stashbot added a comment.Apr 7 2021, 11:39 AM

Mentioned in SAL (#wikimedia-operations) [2021-04-07T11:39:53Z] <marostegui> Deploy schema change on s3 codfw, lag will appear T276150 T276156

• Marostegui updated the task description. (Show Details)Apr 7 2021, 2:39 PM
• Marostegui added a comment.EditedApr 8 2021, 4:59 AM

s3 eqiad

  • labsdb1011
  • labsdb1010
  • labsdb1009
  • dbstore1004
  • db1175
  • db1171
  • db1166
  • db1157
  • db1154
  • db1124
  • db1123
  • db1112
  • clouddb1021
  • clouddb1017
  • clouddb1013
• Marostegui closed this task as Resolved.Apr 13 2021, 8:04 AM
• Marostegui updated the task description. (Show Details)

This is all done

Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL