Page MenuHomePhabricator

ops-monitoring-bot (Operations Monitoring Bot)
UserBot

Projects

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Friday

  • Clear sailing ahead.

User Details

User Since
Aug 12 2016, 1:45 PM (253 w, 4 d)
Roles
Bot
Availability
Available
LDAP User
Unknown
MediaWiki User
Unknown

Bot managed by SRE for automated interaction with Phabricator from monitoring tools.

Recent Activity

Yesterday

ops-monitoring-bot added a comment to T281787: (Need By: TBD) rack/setup/install an-web1001.

Completed auto-reimage of hosts:

['an-web1001.eqiad.wmnet']
Tue, Jun 22, 8:38 PM · Analytics-Clusters, SRE, ops-eqiad, DC-Ops
ops-monitoring-bot added a comment to T281787: (Need By: TBD) rack/setup/install an-web1001.

Script wmf-auto-reimage was launched by robh on cumin1001.eqiad.wmnet for hosts:

an-web1001.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106221955_robh_24556_an-web1001_eqiad_wmnet.log.

Tue, Jun 22, 7:55 PM · Analytics-Clusters, SRE, ops-eqiad, DC-Ops
ops-monitoring-bot added a comment to T281787: (Need By: TBD) rack/setup/install an-web1001.

Completed auto-reimage of hosts:

['an-web1001.eqiad.wmnet']
Tue, Jun 22, 7:06 PM · Analytics-Clusters, SRE, ops-eqiad, DC-Ops
ops-monitoring-bot added a comment to T281787: (Need By: TBD) rack/setup/install an-web1001.

Script wmf-auto-reimage was launched by robh on cumin1001.eqiad.wmnet for hosts:

an-web1001.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106221854_robh_4601_an-web1001_eqiad_wmnet.log.

Tue, Jun 22, 6:54 PM · Analytics-Clusters, SRE, ops-eqiad, DC-Ops
ops-monitoring-bot added a comment to T273914: (Need By: TBD) rack/setup/install thumbor100[56].

Completed auto-reimage of hosts:

['thumbor1005.eqiad.wmnet', 'thumbor1006.eqiad.wmnet']
Tue, Jun 22, 6:38 PM · SRE, serviceops-radar, ops-eqiad, DC-Ops
ops-monitoring-bot added a comment to T273914: (Need By: TBD) rack/setup/install thumbor100[56].

Script wmf-auto-reimage was launched by robh on cumin1001.eqiad.wmnet for hosts:

['thumbor1005.eqiad.wmnet', 'thumbor1006.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106221814_robh_31552.log.

Tue, Jun 22, 6:14 PM · SRE, serviceops-radar, ops-eqiad, DC-Ops
ops-monitoring-bot added a comment to T283235: Upgrade s5 to Debian Buster and MariaDB 10.4.

Completed auto-reimage of hosts:

['db1100.eqiad.wmnet']
Tue, Jun 22, 6:24 AM · Patch-For-Review, DBA
ops-monitoring-bot added a comment to T283235: Upgrade s5 to Debian Buster and MariaDB 10.4.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1100.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106220603_marostegui_5451.log.

Tue, Jun 22, 6:03 AM · Patch-For-Review, DBA
ops-monitoring-bot added a comment to T282603: (Need By: TBD) rack/setup/install ganeti202[56].

Completed auto-reimage of hosts:

['ganeti2026.codfw.wmnet']
Tue, Jun 22, 1:11 AM · SRE, ops-codfw, DC-Ops
ops-monitoring-bot added a comment to T282603: (Need By: TBD) rack/setup/install ganeti202[56].

Script wmf-auto-reimage was launched by pt1979 on cumin2002.codfw.wmnet for hosts:

ganeti2026.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106220046_pt1979_2250535_ganeti2026_codfw_wmnet.log.

Tue, Jun 22, 12:46 AM · SRE, ops-codfw, DC-Ops
ops-monitoring-bot added a comment to T282603: (Need By: TBD) rack/setup/install ganeti202[56].

Completed auto-reimage of hosts:

['ganeti2025.codfw.wmnet']
Tue, Jun 22, 12:44 AM · SRE, ops-codfw, DC-Ops
ops-monitoring-bot added a comment to T282603: (Need By: TBD) rack/setup/install ganeti202[56].

Script wmf-auto-reimage was launched by pt1979 on cumin2002.codfw.wmnet for hosts:

ganeti2025.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106220017_pt1979_2246312_ganeti2025_codfw_wmnet.log.

Tue, Jun 22, 12:17 AM · SRE, ops-codfw, DC-Ops

Mon, Jun 21

ops-monitoring-bot added a comment to T282603: (Need By: TBD) rack/setup/install ganeti202[56].

Completed auto-reimage of hosts:

['ganeti2025.codfw.wmnet']
Mon, Jun 21, 9:30 PM · SRE, ops-codfw, DC-Ops
ops-monitoring-bot added a comment to T282603: (Need By: TBD) rack/setup/install ganeti202[56].

Script wmf-auto-reimage was launched by pt1979 on cumin2002.codfw.wmnet for hosts:

ganeti2025.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106212103_pt1979_2224285_ganeti2025_codfw_wmnet.log.

Mon, Jun 21, 9:04 PM · SRE, ops-codfw, DC-Ops
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin1001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 2:57 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin1001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 2:52 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin1001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 2:47 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin1001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 2:40 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin1001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 2:37 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin1001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 2:34 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T283131: Upgrade s3 to Debian Buster and MariaDB 10.4.

Completed auto-reimage of hosts:

['db1123.eqiad.wmnet']
Mon, Jun 21, 2:31 PM · Patch-For-Review, DBA
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin1001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 2:28 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin1001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 2:22 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin1001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 2:16 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin2001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 2:06 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T283131: Upgrade s3 to Debian Buster and MariaDB 10.4.

Script wmf-auto-reimage was launched by kormat on cumin1001.eqiad.wmnet for hosts:

['db1123.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106211401_kormat_3114.log.

Mon, Jun 21, 2:01 PM · Patch-For-Review, DBA
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin2001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 1:58 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin2001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 1:51 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin2001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 1:45 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin2001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 1:38 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin2001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 1:28 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin2001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 1:21 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin2001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 1:19 PM · Machine-Learning-Team (Active Tasks)
ops-monitoring-bot added a comment to T285225: Update all Liftwing machines to latest packages.

Host rebooted by klausman@cumin2001 with reason: Reboot to activate kernel 4.19.0-17

Mon, Jun 21, 1:14 PM · Machine-Learning-Team (Active Tasks)

Wed, Jun 16

ops-monitoring-bot added a comment to T269582: [OSM] perform imposm3 migration in production.

Completed auto-reimage of hosts:

['maps1007.eqiad.wmnet']
Wed, Jun 16, 12:50 PM · Platform Team Workboards (Platform Engineering Reliability), Patch-For-Review, Product-Infrastructure-Team-Backlog, Maps
ops-monitoring-bot added a comment to T269582: [OSM] perform imposm3 migration in production.

Script wmf-auto-reimage was launched by hnowlan on cumin1001.eqiad.wmnet for hosts:

['maps1007.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106161020_hnowlan_29876.log.

Wed, Jun 16, 10:21 AM · Platform Team Workboards (Platform Engineering Reliability), Patch-For-Review, Product-Infrastructure-Team-Backlog, Maps

Tue, Jun 15

ops-monitoring-bot added a comment to T278423: Upgrade the Hadoop masters to Debian Buster.

Completed auto-reimage of hosts:

['an-master1002.eqiad.wmnet']
Tue, Jun 15, 5:26 PM · User-razzi, Patch-For-Review, Analytics-Kanban, Analytics-Clusters
ops-monitoring-bot added a comment to T278423: Upgrade the Hadoop masters to Debian Buster.

Script wmf-auto-reimage was launched by razzi on cumin1001.eqiad.wmnet for hosts:

an-master1002.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106151655_razzi_16131_an-master1002_eqiad_wmnet.log.

Tue, Jun 15, 4:55 PM · User-razzi, Patch-For-Review, Analytics-Kanban, Analytics-Clusters
ops-monitoring-bot added a comment to T278423: Upgrade the Hadoop masters to Debian Buster.

Icinga downtime set by razzi@cumin1001 for 60 days, 0:00:00 1 host(s) and their services with reason: Update operating system to bullseye

an-master1002.eqiad.wmnet
Tue, Jun 15, 4:11 PM · User-razzi, Patch-For-Review, Analytics-Kanban, Analytics-Clusters

Sat, Jun 12

ops-monitoring-bot created T284852: Degraded RAID on db2148.
Sat, Jun 12, 5:38 AM · DBA, SRE, ops-codfw

Fri, Jun 11

ops-monitoring-bot created T284849: Degraded RAID on pc2014.
Fri, Jun 11, 10:30 PM · DBA, SRE, ops-codfw
ops-monitoring-bot added a comment to T282482: (Need By: TBD) rack/setup/install pc2011-pc2014.

Completed auto-reimage of hosts:

['pc2014.codfw.wmnet']
Fri, Jun 11, 9:37 PM · Patch-For-Review, SRE, ops-codfw, DC-Ops
ops-monitoring-bot added a comment to T282482: (Need By: TBD) rack/setup/install pc2011-pc2014.

Script wmf-auto-reimage was launched by pt1979 on cumin2002.codfw.wmnet for hosts:

pc2014.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106112110_pt1979_620252_pc2014_codfw_wmnet.log.

Fri, Jun 11, 9:10 PM · Patch-For-Review, SRE, ops-codfw, DC-Ops
ops-monitoring-bot added a comment to T282482: (Need By: TBD) rack/setup/install pc2011-pc2014.

Completed auto-reimage of hosts:

['pc2013.codfw.wmnet']
Fri, Jun 11, 9:10 PM · Patch-For-Review, SRE, ops-codfw, DC-Ops
ops-monitoring-bot created T284845: Degraded RAID on pc2012.
Fri, Jun 11, 9:00 PM · DBA, SRE, ops-codfw
ops-monitoring-bot added a comment to T282482: (Need By: TBD) rack/setup/install pc2011-pc2014.

Script wmf-auto-reimage was launched by pt1979 on cumin2002.codfw.wmnet for hosts:

pc2013.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106112044_pt1979_615456_pc2013_codfw_wmnet.log.

Fri, Jun 11, 8:44 PM · Patch-For-Review, SRE, ops-codfw, DC-Ops
ops-monitoring-bot added a comment to T282482: (Need By: TBD) rack/setup/install pc2011-pc2014.

Completed auto-reimage of hosts:

['pc2012.codfw.wmnet']
Fri, Jun 11, 8:13 PM · Patch-For-Review, SRE, ops-codfw, DC-Ops
ops-monitoring-bot added a comment to T282482: (Need By: TBD) rack/setup/install pc2011-pc2014.

Script wmf-auto-reimage was launched by pt1979 on cumin2002.codfw.wmnet for hosts:

pc2012.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106111947_pt1979_607838_pc2012_codfw_wmnet.log.

Fri, Jun 11, 7:47 PM · Patch-For-Review, SRE, ops-codfw, DC-Ops
ops-monitoring-bot added a comment to T282482: (Need By: TBD) rack/setup/install pc2011-pc2014.

Completed auto-reimage of hosts:

['pc2011.codfw.wmnet']
Fri, Jun 11, 7:36 PM · Patch-For-Review, SRE, ops-codfw, DC-Ops
ops-monitoring-bot added a comment to T282482: (Need By: TBD) rack/setup/install pc2011-pc2014.

Script wmf-auto-reimage was launched by pt1979 on cumin2002.codfw.wmnet for hosts:

pc2011.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106111910_pt1979_602832_pc2011_codfw_wmnet.log.

Fri, Jun 11, 7:10 PM · Patch-For-Review, SRE, ops-codfw, DC-Ops

Thu, Jun 10

ops-monitoring-bot created T284719: Degraded RAID on ms-be2038.
Thu, Jun 10, 6:22 AM · SRE, ops-codfw
ops-monitoring-bot created T284718: Degraded RAID on ms-be2038.
Thu, Jun 10, 5:39 AM · SRE, ops-codfw
ops-monitoring-bot created T284714: Degraded RAID on ms-be2038.
Thu, Jun 10, 2:36 AM · SRE, ops-codfw
ops-monitoring-bot created T284713: Degraded RAID on ms-be2038.
Thu, Jun 10, 1:21 AM · SRE, ops-codfw
ops-monitoring-bot created T284710: Degraded RAID on ms-be2038.
Thu, Jun 10, 12:05 AM · SRE, ops-codfw

Wed, Jun 9

ops-monitoring-bot created T284709: Degraded RAID on ms-be2038.
Wed, Jun 9, 10:50 PM · SRE, ops-codfw
ops-monitoring-bot created T284690: Degraded RAID on ms-be2038.
Wed, Jun 9, 7:14 PM · SRE, ops-codfw
ops-monitoring-bot created T284682: Degraded RAID on ms-be2038.
Wed, Jun 9, 5:59 PM · SRE, ops-codfw
ops-monitoring-bot added a comment to T269582: [OSM] perform imposm3 migration in production.

Completed auto-reimage of hosts:

['maps2009.codfw.wmnet']
Wed, Jun 9, 4:24 PM · Platform Team Workboards (Platform Engineering Reliability), Patch-For-Review, Product-Infrastructure-Team-Backlog, Maps
ops-monitoring-bot added a comment to T269582: [OSM] perform imposm3 migration in production.

Script wmf-auto-reimage was launched by hnowlan on cumin2001.codfw.wmnet for hosts:

['maps2009.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106091536_hnowlan_32077.log.

Wed, Jun 9, 3:36 PM · Platform Team Workboards (Platform Engineering Reliability), Patch-For-Review, Product-Infrastructure-Team-Backlog, Maps
ops-monitoring-bot added a comment to T283235: Upgrade s5 to Debian Buster and MariaDB 10.4.

Completed auto-reimage of hosts:

['db1130.eqiad.wmnet']
Wed, Jun 9, 10:29 AM · Patch-For-Review, DBA
ops-monitoring-bot added a comment to T283235: Upgrade s5 to Debian Buster and MariaDB 10.4.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1130.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106091007_marostegui_1368.log.

Wed, Jun 9, 10:07 AM · Patch-For-Review, DBA

Tue, Jun 8

ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Completed auto-reimage of hosts:

['wdqs1009.eqiad.wmnet']
Tue, Jun 8, 9:54 PM · User-jbond, Discovery-Search (Current work)
ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Script wmf-auto-reimage was launched by ryankemper on cumin1001.eqiad.wmnet for hosts:

wdqs1009.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106082128_ryankemper_20112_wdqs1009_eqiad_wmnet.log.

Tue, Jun 8, 9:28 PM · User-jbond, Discovery-Search (Current work)
ops-monitoring-bot added a comment to T283235: Upgrade s5 to Debian Buster and MariaDB 10.4.

Completed auto-reimage of hosts:

['db2123.codfw.wmnet']
Tue, Jun 8, 5:56 AM · Patch-For-Review, DBA
ops-monitoring-bot added a comment to T283235: Upgrade s5 to Debian Buster and MariaDB 10.4.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db2123.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106080527_marostegui_5778.log.

Tue, Jun 8, 5:27 AM · Patch-For-Review, DBA
ops-monitoring-bot added a comment to T283235: Upgrade s5 to Debian Buster and MariaDB 10.4.

Completed auto-reimage of hosts:

['db2123.codfw.wmnet']
Tue, Jun 8, 5:26 AM · Patch-For-Review, DBA
ops-monitoring-bot added a comment to T283235: Upgrade s5 to Debian Buster and MariaDB 10.4.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db2123.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106080459_marostegui_32562.log.

Tue, Jun 8, 4:59 AM · Patch-For-Review, DBA

Mon, Jun 7

ops-monitoring-bot added a comment to T283131: Upgrade s3 to Debian Buster and MariaDB 10.4.

Completed auto-reimage of hosts:

['db1157.eqiad.wmnet']
Mon, Jun 7, 10:46 AM · Patch-For-Review, DBA
ops-monitoring-bot added a comment to T283131: Upgrade s3 to Debian Buster and MariaDB 10.4.

Script wmf-auto-reimage was launched by kormat on cumin1001.eqiad.wmnet for hosts:

['db1157.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106071021_kormat_12649.log.

Mon, Jun 7, 10:21 AM · Patch-For-Review, DBA
ops-monitoring-bot added a comment to T283235: Upgrade s5 to Debian Buster and MariaDB 10.4.

Completed auto-reimage of hosts:

['db2113.codfw.wmnet']
Mon, Jun 7, 5:38 AM · Patch-For-Review, DBA
ops-monitoring-bot added a comment to T283235: Upgrade s5 to Debian Buster and MariaDB 10.4.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db2113.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106070512_marostegui_17172.log.

Mon, Jun 7, 5:12 AM · Patch-For-Review, DBA

Fri, Jun 4

ops-monitoring-bot added a comment to T278729: cp1087 down with hardware issues.

Completed auto-reimage of hosts:

['cp1087.eqiad.wmnet']
Fri, Jun 4, 8:56 PM · ops-eqiad, SRE, Traffic
ops-monitoring-bot added a comment to T278729: cp1087 down with hardware issues.

Script wmf-auto-reimage was launched by bblack on cumin1001.eqiad.wmnet for hosts:

cp1087.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106041952_bblack_19504_cp1087_eqiad_wmnet.log.

Fri, Jun 4, 7:52 PM · ops-eqiad, SRE, Traffic
ops-monitoring-bot added a comment to T281399: cloudvirt1040 primary NIC disconnected.

Completed auto-reimage of hosts:

['cloudvirt1040.eqiad.wmnet']
Fri, Jun 4, 6:34 PM · cloud-services-team (Hardware), SRE, ops-eqiad
ops-monitoring-bot added a comment to T281399: cloudvirt1040 primary NIC disconnected.

Script wmf-auto-reimage was launched by andrew on cumin1001.eqiad.wmnet for hosts:

['cloudvirt1040.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106041710_andrew_7137.log.

Fri, Jun 4, 5:10 PM · cloud-services-team (Hardware), SRE, ops-eqiad
ops-monitoring-bot added a comment to T281399: cloudvirt1040 primary NIC disconnected.

Script wmf-auto-reimage was launched by andrew on cumin1001.eqiad.wmnet for hosts:

['cloudvirt1040.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106041643_andrew_31194.log.

Fri, Jun 4, 4:44 PM · cloud-services-team (Hardware), SRE, ops-eqiad
ops-monitoring-bot added a comment to T281399: cloudvirt1040 primary NIC disconnected.

Script wmf-auto-reimage was launched by andrew on cumin1001.eqiad.wmnet for hosts:

['cloudvirt1040.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106041638_andrew_30711.log.

Fri, Jun 4, 4:38 PM · cloud-services-team (Hardware), SRE, ops-eqiad
ops-monitoring-bot added a comment to T281399: cloudvirt1040 primary NIC disconnected.

Completed auto-reimage of hosts:

['cloudvirt1040.eqiad.wmnet']
Fri, Jun 4, 4:34 PM · cloud-services-team (Hardware), SRE, ops-eqiad
ops-monitoring-bot added a comment to T281399: cloudvirt1040 primary NIC disconnected.

Script wmf-auto-reimage was launched by andrew on cumin1001.eqiad.wmnet for hosts:

['cloudvirt1040.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106041532_andrew_20422.log.

Fri, Jun 4, 3:32 PM · cloud-services-team (Hardware), SRE, ops-eqiad
ops-monitoring-bot added a comment to T281399: cloudvirt1040 primary NIC disconnected.

Script wmf-auto-reimage was launched by andrew on cumin1001.eqiad.wmnet for hosts:

['cloudvirt1040.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106041443_andrew_14472.log.

Fri, Jun 4, 2:43 PM · cloud-services-team (Hardware), SRE, ops-eqiad
ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Completed auto-reimage of hosts:

['wdqs2002.codfw.wmnet']
Fri, Jun 4, 4:54 AM · User-jbond, Discovery-Search (Current work)
ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Script wmf-auto-reimage was launched by ryankemper on cumin2002.codfw.wmnet for hosts:

wdqs2002.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106040424_ryankemper_3238923_wdqs2002_codfw_wmnet.log.

Fri, Jun 4, 4:25 AM · User-jbond, Discovery-Search (Current work)

Thu, Jun 3

ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Completed auto-reimage of hosts:

['wdqs2001.codfw.wmnet']
Thu, Jun 3, 11:42 PM · User-jbond, Discovery-Search (Current work)
ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Script wmf-auto-reimage was launched by ryankemper on cumin2002.codfw.wmnet for hosts:

wdqs2001.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106032241_ryankemper_3197307_wdqs2001_codfw_wmnet.log.

Thu, Jun 3, 11:11 PM · User-jbond, Discovery-Search (Current work)
ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Completed auto-reimage of hosts:

['wdqs2005.codfw.wmnet']
Thu, Jun 3, 7:15 PM · User-jbond, Discovery-Search (Current work)
ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Completed auto-reimage of hosts:

['wdqs1005.eqiad.wmnet']
Thu, Jun 3, 7:12 PM · User-jbond, Discovery-Search (Current work)
ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Script wmf-auto-reimage was launched by ryankemper on cumin2002.codfw.wmnet for hosts:

wdqs2005.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106031846_ryankemper_3168105_wdqs2005_codfw_wmnet.log.

Thu, Jun 3, 6:47 PM · User-jbond, Discovery-Search (Current work)
ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Script wmf-auto-reimage was launched by ryankemper on cumin1001.eqiad.wmnet for hosts:

wdqs1005.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106031846_ryankemper_5874_wdqs1005_eqiad_wmnet.log.

Thu, Jun 3, 6:47 PM · User-jbond, Discovery-Search (Current work)
ops-monitoring-bot added a comment to T276637: (Need By: TBD) rack/setup/install moss-be100[12].

Completed auto-reimage of hosts:

['moss-be1001.eqiad.wmnet', 'moss-be1002.eqiad.wmnet']
Thu, Jun 3, 5:58 PM · SRE, ops-eqiad, DC-Ops
ops-monitoring-bot added a comment to T276637: (Need By: TBD) rack/setup/install moss-be100[12].

Script wmf-auto-reimage was launched by robh on cumin1001.eqiad.wmnet for hosts:

['moss-be1001.eqiad.wmnet', 'moss-be1002.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106031731_robh_16029.log.

Thu, Jun 3, 5:31 PM · SRE, ops-eqiad, DC-Ops
ops-monitoring-bot added a comment to T276637: (Need By: TBD) rack/setup/install moss-be100[12].

Completed auto-reimage of hosts:

['moss-be1001.eqiad.wmnet', 'moss-be1002.eqiad.wmnet']
Thu, Jun 3, 5:26 PM · SRE, ops-eqiad, DC-Ops
ops-monitoring-bot added a comment to T276637: (Need By: TBD) rack/setup/install moss-be100[12].

Script wmf-auto-reimage was launched by robh on cumin1001.eqiad.wmnet for hosts:

['moss-be1001.eqiad.wmnet', 'moss-be1002.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106031710_robh_31308.log.

Thu, Jun 3, 5:10 PM · SRE, ops-eqiad, DC-Ops
ops-monitoring-bot added a comment to T276637: (Need By: TBD) rack/setup/install moss-be100[12].

Script wmf-auto-reimage was launched by robh on cumin1001.eqiad.wmnet for hosts:

['moss-be1001.eqiad.wmnet', 'moss-be1002.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202106031527_robh_2852.log.

Thu, Jun 3, 3:28 PM · SRE, ops-eqiad, DC-Ops
ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Completed auto-reimage of hosts:

['wdqs1008.eqiad.wmnet']
Thu, Jun 3, 2:32 AM · User-jbond, Discovery-Search (Current work)
ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Completed auto-reimage of hosts:

['wdqs2006.codfw.wmnet']
Thu, Jun 3, 2:19 AM · User-jbond, Discovery-Search (Current work)
ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Script wmf-auto-reimage was launched by ryankemper on cumin1001.eqiad.wmnet for hosts:

wdqs1008.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106030207_ryankemper_3293_wdqs1008_eqiad_wmnet.log.

Thu, Jun 3, 2:07 AM · User-jbond, Discovery-Search (Current work)
ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Script wmf-auto-reimage was launched by ryankemper on cumin2002.codfw.wmnet for hosts:

wdqs2006.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106030150_ryankemper_3011426_wdqs2006_codfw_wmnet.log.

Thu, Jun 3, 1:51 AM · User-jbond, Discovery-Search (Current work)

Wed, Jun 2

ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Completed auto-reimage of hosts:

['wdqs2003.codfw.wmnet']
Wed, Jun 2, 11:07 PM · User-jbond, Discovery-Search (Current work)
ops-monitoring-bot added a comment to T280382: WDQS hosts low on /srv disk space.

Script wmf-auto-reimage was launched by ryankemper on cumin2002.codfw.wmnet for hosts:

wdqs2003.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202106022239_ryankemper_2989124_wdqs2003_codfw_wmnet.log.

Wed, Jun 2, 10:39 PM · User-jbond, Discovery-Search (Current work)