Page MenuHomePhabricator

Setup newer machines and replace all old misc (m*) and x1 codfw machines
Closed, ResolvedPublic

Description

db2078 was already setup, although probably is too large to stay like that.
x1 host was setup, but there is not redundancy for it on codfw.

Pending to replace:

  • db2011
  • db2012
  • db2030

New hosts to setup:

  • s5 replica (or setup redundancy on eqiad only)

Related Objects

Event Timeline

jcrespo triaged this task as Medium priority.Dec 21 2017, 3:06 PM
jcrespo created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 21 2017, 3:06 PM

Eqiad version: T183469
Very related: T170662

Change 399782 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/mediawiki-config@master] mariadb: Repool db1055 & db1056 as x1 replicas

https://gerrit.wikimedia.org/r/399782

Marostegui moved this task from Triage to In progress on the DBA board.Dec 27 2017, 7:24 AM

Change 399782 merged by jenkins-bot:
[operations/mediawiki-config@master] mariadb: Repool db1055 & db1056 as x1 replicas

https://gerrit.wikimedia.org/r/399782

Marostegui moved this task from In progress to Next on the DBA board.Feb 9 2018, 1:16 PM

Change 410790 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/mediawiki-config@master] mariadb: Remove db2042 from mediawiki

https://gerrit.wikimedia.org/r/410790

Change 410794 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Move db2042 from codfw:core:s1 to misc:s3 (phabricator)

https://gerrit.wikimedia.org/r/410794

Change 410790 merged by Jcrespo:
[operations/mediawiki-config@master] mariadb: Remove db2042 from mediawiki

https://gerrit.wikimedia.org/r/410790

Change 411198 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Prepare reimage of db1053 and db2042 to strech

https://gerrit.wikimedia.org/r/411198

Change 411198 merged by Jcrespo:
[operations/puppet@production] mariadb: Prepare reimage of db1053 and db2042 to strech

https://gerrit.wikimedia.org/r/411198

Script wmf-auto-reimage was launched by jynus on sarin.codfw.wmnet for hosts:

['db2042.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201802161034_jynus_12767.log.

Change 410794 merged by Jcrespo:
[operations/puppet@production] mariadb: Move db2042 from codfw:core:s1 to misc:s3 (phabricator)

https://gerrit.wikimedia.org/r/410794

Completed auto-reimage of hosts:

['db2042.codfw.wmnet']

and were ALL successful.

Change 411201 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Disable notifications for db1043 and db2012

https://gerrit.wikimedia.org/r/411201

Change 411201 merged by Jcrespo:
[operations/puppet@production] mariadb: Disable notifications for db1043 and db2012

https://gerrit.wikimedia.org/r/411201

Change 411203 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Change phabricator db config to modern defaults

https://gerrit.wikimedia.org/r/411203

Change 411203 merged by Jcrespo:
[operations/puppet@production] mariadb: Change phabricator db config to modern defaults

https://gerrit.wikimedia.org/r/411203

Change 411216 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] Mariadb: Set default basedir for phabricator (Mariadb 10.1)

https://gerrit.wikimedia.org/r/411216

Change 411216 merged by Jcrespo:
[operations/puppet@production] Mariadb: Set default basedir for phabricator (Mariadb 10.1)

https://gerrit.wikimedia.org/r/411216

Change 412978 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Allow db2044 reimage on stretch

https://gerrit.wikimedia.org/r/412978

Change 412978 merged by Jcrespo:
[operations/puppet@production] mariadb: Allow db2044 reimage on stretch

https://gerrit.wikimedia.org/r/412978

Change 412994 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Move db2044 from codfw-core-s4 to codfw-misc-s2

https://gerrit.wikimedia.org/r/412994

Change 412995 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] dbproxy: Setup db2044 as the main m2 host on codfw and monitor it

https://gerrit.wikimedia.org/r/412995

Script wmf-auto-reimage was launched by jynus on sarin.codfw.wmnet for hosts:

['db2044.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201802211042_jynus_5930.log.

Change 412994 merged by Jcrespo:
[operations/puppet@production] mariadb: Move db2044 from codfw-core-s4 to codfw-misc-s2

https://gerrit.wikimedia.org/r/412994

Completed auto-reimage of hosts:

['db2044.codfw.wmnet']

and were ALL successful.

Change 412995 merged by Jcrespo:
[operations/puppet@production] dbproxy: Setup db2044 as the main m2 host on codfw and monitor it

https://gerrit.wikimedia.org/r/412995

Change 413159 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Reenable notifications on db2044 (m2)

https://gerrit.wikimedia.org/r/413159

Change 413167 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb: Update socket location of misc services (m1, m2, m5)

https://gerrit.wikimedia.org/r/413167

Change 413159 merged by Jcrespo:
[operations/puppet@production] mariadb: Reenable notifications on db2044 (m2)

https://gerrit.wikimedia.org/r/413159

I would say this is resolved- only pending the actual decommission (tracked on separate tickets), and the setup of extra servers for redundancy (tracked on T170662).

jcrespo closed this task as Resolved.Feb 21 2018, 3:30 PM
jcrespo claimed this task.

Change 413167 abandoned by Jcrespo:
mariadb: Update socket location of misc services (m1, m2, m5)

Reason:
Already merged elsewhere.

https://gerrit.wikimedia.org/r/413167