Page MenuHomePhabricator

Productionize db213[6-9] and db2140
Open, MediumPublic

Description

The following hosts need to be productionized in s4:

  • db2136 will replace db2073 (sanitarium master)
    • db2136 pooled in s4
    • change db2095 to replicate from db2136
  • db2137 will replace db2084 (multi instance with s4 and s5)
  • db2138 will replace db2091 (multi instance with s4 and s2)
  • db2139 will replace db2099 (multi instance with s4 and s5)
    • db2139 pooled in s4+s5
    • move backups to use db2139 instead of db2099
  • db2140 will replace db2090 (master)

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMon, May 18, 5:21 AM
Marostegui triaged this task as Medium priority.Mon, May 18, 5:21 AM
Marostegui moved this task from Triage to Next on the DBA board.
Marostegui updated the task description. (Show Details)Mon, May 18, 9:15 AM
Marostegui assigned this task to Kormat.Mon, May 18, 9:32 AM
Marostegui moved this task from Next to In progress on the DBA board.

Change 597050 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] Add db2136 to s4

https://gerrit.wikimedia.org/r/597050

Change 597050 merged by Kormat:
[operations/puppet@production] mariadb: Add db2136 to s4

https://gerrit.wikimedia.org/r/597050

Mentioned in SAL (#wikimedia-operations) [2020-05-18T14:15:05Z] <kormat@cumin1001> dbctl commit (dc=all): 'Depool db2073 while replacing it T252985', diff saved to https://phabricator.wikimedia.org/P11216 and previous config saved to /var/cache/conftool/dbconfig/20200518-141505-kormat.json

Change 597224 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] mariadb: Remove db2136 from spare role.

https://gerrit.wikimedia.org/r/597224

Change 597224 merged by Kormat:
[operations/puppet@production] mariadb: Remove db2136 from spare role.

https://gerrit.wikimedia.org/r/597224

Change 597273 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] hieradata: enable notifications for db2136

https://gerrit.wikimedia.org/r/597273

Change 597273 merged by Kormat:
[operations/puppet@production] hieradata: enable notifications for db2136

https://gerrit.wikimedia.org/r/597273

Mentioned in SAL (#wikimedia-operations) [2020-05-19T13:03:14Z] <kormat@cumin1001> dbctl commit (dc=all): 'Pool db2136 into s4 T252985', diff saved to https://phabricator.wikimedia.org/P11233 and previous config saved to /var/cache/conftool/dbconfig/20200519-130313-kormat.json

Kormat updated the task description. (Show Details)Tue, May 19, 1:06 PM
Kormat updated the task description. (Show Details)

Mentioned in SAL (#wikimedia-operations) [2020-05-19T15:23:41Z] <kormat@cumin1001> dbctl commit (dc=all): 'Repool db2073 into s4 T252985', diff saved to https://phabricator.wikimedia.org/P11236 and previous config saved to /var/cache/conftool/dbconfig/20200519-152340-kormat.json

db2137 is set up in s4+s5, and it's currently restoring logical backups from dbprov2001.

Change 598061 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] mariadb: Enable notifications for db2137

https://gerrit.wikimedia.org/r/598061

Change 598061 merged by Kormat:
[operations/puppet@production] mariadb: Enable notifications for db2137

https://gerrit.wikimedia.org/r/598061

Mentioned in SAL (#wikimedia-operations) [2020-05-22T15:01:21Z] <kormat@cumin1001> dbctl commit (dc=all): 'Pool db2137 into s4+s5 T252985', diff saved to https://phabricator.wikimedia.org/P11292 and previous config saved to /var/cache/conftool/dbconfig/20200522-150120-kormat.json

Kormat updated the task description. (Show Details)Fri, May 22, 3:01 PM

Mentioned in SAL (#wikimedia-operations) [2020-05-25T11:27:20Z] <marostegui> Extend /srv 1100G on db213[6-9] T252985

Change 598736 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] mariadb: Add db2138 to s2+s4

https://gerrit.wikimedia.org/r/598736

Change 598736 merged by Kormat:
[operations/puppet@production] mariadb: Add db2138 to s2+s4

https://gerrit.wikimedia.org/r/598736

db2138 is set up, and currently catching up on replication before it gets pooled into s2+s4.

Change 599304 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] mariadb: Enable notifications for db2138

https://gerrit.wikimedia.org/r/599304

Change 599304 merged by Kormat:
[operations/puppet@production] mariadb: Enable notifications for db2138

https://gerrit.wikimedia.org/r/599304

Mentioned in SAL (#wikimedia-operations) [2020-05-28T11:03:34Z] <kormat@cumin1001> dbctl commit (dc=all): 'Add db2138 to s2+s4 T252985', diff saved to https://phabricator.wikimedia.org/P11330 and previous config saved to /var/cache/conftool/dbconfig/20200528-110333-kormat.json

Kormat updated the task description. (Show Details)Thu, May 28, 12:03 PM

Change 599313 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] mariadb: Add db2139 to s4+s5

https://gerrit.wikimedia.org/r/599313

Change 599313 merged by Kormat:
[operations/puppet@production] mariadb: Add db2139 to s4+s5

https://gerrit.wikimedia.org/r/599313

Change 599739 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] mariadb: Enable notifications for db2139

https://gerrit.wikimedia.org/r/599739

Change 599746 had a related patch set uploaded (by Kormat; owner: Kormat):
[operations/puppet@production] maraidb: Add db2040 to s4

https://gerrit.wikimedia.org/r/599746

Change 599739 merged by Kormat:
[operations/puppet@production] mariadb: Enable notifications for db2139

https://gerrit.wikimedia.org/r/599739

Kormat updated the task description. (Show Details)Fri, May 29, 9:48 AM