Page MenuHomePhabricator

Productionize db1206-db1225
Open, MediumPublic

Description

db1206-db1225 will replace db1106-db1125

  • db1206 to replace db1106 (T322256)
  • db1207 to replace db1107
  • db1208 to replace db1108 (Analytics - will create an specific task for this)
  • db1209 to replace db1109
  • db1210 to replace db1110
  • db1211 to replace db1111
  • db1212 to replace db1112
  • db1213 to replace db1113
  • db1214 to replace db1114
  • db1215 to replace db1115
  • db1216 to replace db1116
  • db1217 to replace db1117
  • db1218 to replace db1118
  • db1219 to replace db1119
  • db1220 to replace db1120
  • db1221 to replace db1121
  • db1222 to replace db1122
  • db1223 to replace db1123
  • db1224 to replace db1124
  • db1225 to replace db1125

Event Timeline

Marostegui triaged this task as Medium priority.Tue, Jan 10, 7:35 PM
Marostegui moved this task from Triage to Blocked on the DBA board.
Marostegui renamed this task from Productionize db1207-db1225 to Productionize db1206-db1225.Tue, Jan 10, 9:51 PM
Marostegui updated the task description. (Show Details)

db1206 was already productionized. However, as it needs to be an exact copy of db1106 (s1 sanitarium master - I will reclone it from that host to avoid any unexpected surprises data-wise)

Change 878202 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] db1206: No longer testing RAID controller

https://gerrit.wikimedia.org/r/878202

Change 878202 merged by Marostegui:

[operations/puppet@production] db1206: No longer testing RAID controller

https://gerrit.wikimedia.org/r/878202

Mentioned in SAL (#wikimedia-operations) [2023-01-11T15:21:24Z] <marostegui> Stop mariadb on db1106 to reclone db1206 (there will be lag on s1 on wikireplicas) T326669

db1206 was already productionized. However, as it needs to be an exact copy of db1106 (s1 sanitarium master - I will reclone it from that host to avoid any unexpected surprises data-wise)

This has been done.
I am going to let db1206 run for a few days before switching sanitarium to replicate from that host.

Repooling db1206 but still NOT moving the wikireplicas under it - that will happen next week.

Mentioned in SAL (#wikimedia-operations) [2023-01-23T07:13:23Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1106 db1206 T326669', diff saved to https://phabricator.wikimedia.org/P43211 and previous config saved to /var/cache/conftool/dbconfig/20230123-071323-marostegui.json

Change 882515 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] mariadb: Switch s1 sanitarium master

https://gerrit.wikimedia.org/r/882515

Change 882515 merged by Marostegui:

[operations/puppet@production] mariadb: Switch s1 sanitarium master

https://gerrit.wikimedia.org/r/882515

I have switched s1 sanitarium master. db1154:3311 now replicates from db1206. I am going to wait a couple of days before decommissioning db1106

Mentioned in SAL (#wikimedia-operations) [2023-01-23T08:42:40Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Add db1206 to vslow and dump group T326669', diff saved to https://phabricator.wikimedia.org/P43228 and previous config saved to /var/cache/conftool/dbconfig/20230123-084239-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2023-01-23T08:43:27Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Add db1206 to vslow and dump group T326669', diff saved to https://phabricator.wikimedia.org/P43229 and previous config saved to /var/cache/conftool/dbconfig/20230123-084326-marostegui.json

Change 883141 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] site.pp: Add db1206 as sanitarium master

https://gerrit.wikimedia.org/r/883141

Change 883141 merged by Marostegui:

[operations/puppet@production] site.pp: Add db1206 as sanitarium master

https://gerrit.wikimedia.org/r/883141