Page MenuHomePhabricator

Reboot dbproxy for kernel upgrades
Closed, ResolvedPublic

Description

  • dbproxy1012.eqiad.wmnet
  • dbproxy1013.eqiad.wmnet
  • dbproxy1014.eqiad.wmnet
  • dbproxy1015.eqiad.wmnet
  • dbproxy1016.eqiad.wmnet
  • dbproxy1017.eqiad.wmnet
  • dbproxy1020.eqiad.wmnet
  • dbproxy1021.eqiad.wmnet
  • dbproxy2001.codfw.wmnet
  • dbproxy2002.codfw.wmnet
  • dbproxy2003.codfw.wmnet
  • dbproxy2004.codfw.wmnet

Related Objects

StatusSubtypeAssignedTask
ResolvedMarostegui

Event Timeline

Marostegui created this task.
Marostegui moved this task from Triage to Ready on the DBA board.

Change 805095 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] dbproxy2*: Disable notifications

https://gerrit.wikimedia.org/r/805095

Change 805095 merged by Marostegui:

[operations/puppet@production] dbproxy2*: Disable notifications

https://gerrit.wikimedia.org/r/805095

Active proxies:

# for i in m1 m2 m3 m5; do host $i-master | grep alias ;done
m1-master.eqiad.wmnet is an alias for dbproxy1012.eqiad.wmnet.
m2-master.eqiad.wmnet is an alias for dbproxy1015.eqiad.wmnet.
m3-master.eqiad.wmnet is an alias for dbproxy1016.eqiad.wmnet.
m5-master.eqiad.wmnet is an alias for dbproxy1021.eqiad.wmnet.

Change 805114 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/dns@master] wmnet: Failover m1 and m2

https://gerrit.wikimedia.org/r/805114

Change 805114 merged by Marostegui:

[operations/dns@master] wmnet: Failover m1 and m2

https://gerrit.wikimedia.org/r/805114

Mentioned in SAL (#wikimedia-operations) [2022-06-13T14:42:53Z] <marostegui> Failover m1 and m2 to a different proxy T310484

Mentioned in SAL (#wikimedia-operations) [2022-06-14T06:27:35Z] <marostegui> Reboot dbproxy1012 and dbproxy1015 T310484

Change 805271 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/dns@master] wmnet: Failover m3 and m5 master.

https://gerrit.wikimedia.org/r/805271

Change 805271 merged by Marostegui:

[operations/dns@master] wmnet: Failover m3 and m5 master.

https://gerrit.wikimedia.org/r/805271

m3 and m5 dbproxies failed over, waiting a few hours for connections to move gracefully to the other proxies

Mentioned in SAL (#wikimedia-operations) [2022-06-15T05:03:17Z] <marostegui> Reboot dbproxy1016 and dbproxy1021 T310484

Marostegui updated the task description. (Show Details)

All done

===== NODE GROUP =====
(12) dbproxy[2001-2004].codfw.wmnet,dbproxy[1012-1017,1020-1021].eqiad.wmnet
----- OUTPUT of 'sudo uname -v' -----
#1 SMP Debian 5.10.120-1 (2022-06-09)

dbproxy1018 and dbproxy1019 are owned by WMCS