Page MenuHomePhabricator

es1019 mgmt interface DOWN
Closed, ResolvedPublic

Description

es1019 has reported its mgmt iface down.

˜/icinga-wm 7:05> PROBLEM - Host es1019.mgmt is DOWN: PING CRITICAL - Packet loss = 100%

And it looks down indeed:

root@neodymium:~# ping es1019.mgmt.eqiad.wmnet
PING es1019.mgmt.eqiad.wmnet (10.65.4.44) 56(84) bytes of data.

If you need to reboot the host, let us know so we can depool it first.

Event Timeline

Marostegui triaged this task as Medium priority.Aug 3 2018, 5:15 AM

Change 450912 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/mediawiki-config@master] mariadb: Depool es1019 for hardware maintenance

https://gerrit.wikimedia.org/r/450912

Change 450912 merged by jenkins-bot:
[operations/mediawiki-config@master] mariadb: Depool es1019 for hardware maintenance

https://gerrit.wikimedia.org/r/450912

Mentioned in SAL (#wikimedia-operations) [2018-08-07T10:19:14Z] <jynus> shutting down es1019 for hw maintenance T201132

@Cmjohnson es1019 is fully depooled, alerts disabled and shutdown, please proceed directly with any task you need to do it, ping us here when finished.

Change 450990 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/mediawiki-config@master] mariadb: Repool es1019 and db1102 with low load after maintenance

https://gerrit.wikimedia.org/r/450990

Change 450990 merged by jenkins-bot:
[operations/mediawiki-config@master] mariadb: Repool es1019 and db1102 with low load after maintenance

https://gerrit.wikimedia.org/r/450990

Change 450994 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/mediawiki-config@master] mariadb: Repool db1081 with low load after maintenance

https://gerrit.wikimedia.org/r/450994

Change 450994 merged by jenkins-bot:
[operations/mediawiki-config@master] mariadb: Repool db1081 with low load after maintenance

https://gerrit.wikimedia.org/r/450994

Everything looking ok now.