Page MenuHomePhabricator

ops-monitoring-bot (Operations Monitoring Bot)
UserBot

Projects

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Saturday

  • Clear sailing ahead.

User Details

User Since
Aug 12 2016, 1:45 PM (144 w, 5 d)
Roles
Bot
Availability
Available
LDAP User
Unknown
MediaWiki User
Unknown

Bot managed by Operations for automated interaction with Phabricator from monitoring tools.

Recent Activity

Yesterday

ops-monitoring-bot added a comment to T222937: Replace Varnish backends with ATS on cache upload nodes in esams.

Completed auto-reimage of hosts:

['cp3046.esams.wmnet']
Wed, May 22, 3:39 PM · Patch-For-Review, Operations, Traffic
ops-monitoring-bot added a comment to T222937: Replace Varnish backends with ATS on cache upload nodes in esams.

Script wmf-auto-reimage was launched by bblack on cumin1001.eqiad.wmnet for hosts:

['cp3046.esams.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905221401_bblack_31062.log.

Wed, May 22, 2:01 PM · Patch-For-Review, Operations, Traffic
ops-monitoring-bot added a comment to T221533: Decommission old coredb machines (<=db2042).

Completed auto-reimage of hosts:

['db2070.codfw.wmnet']
Wed, May 22, 9:40 AM · DBA
ops-monitoring-bot added a comment to T221533: Decommission old coredb machines (<=db2042).

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db2070.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905220911_marostegui_234415.log.

Wed, May 22, 9:11 AM · DBA
ops-monitoring-bot added projects to T224087: Degraded RAID on sulfur: Operations, ops-eqiad.
Wed, May 22, 8:44 AM · ops-eqiad, Operations

Tue, May 21

ops-monitoring-bot added a comment to T224023: Decommission analytics-tool1003 (old superset host).

cookbooks.sre.hosts.decommission executed by elukey@cumin1001 for hosts: analytics-tool1003.eqiad.wmnet

  • analytics-tool1003.eqiad.wmnet
    • Removed from Puppet master and PuppetDB
    • Downtimed host on Icinga
    • No management interface found (likely a VM)
    • Removed from DebMonitor
Tue, May 21, 2:25 PM · Patch-For-Review, Operations, vm-requests, Analytics-Kanban, User-Elukey, Analytics

Sun, May 19

ops-monitoring-bot added projects to T223825: Degraded RAID on restbase-dev1006: Operations, ops-eqiad.
Sun, May 19, 1:42 PM · ops-eqiad, Operations
ops-monitoring-bot added a comment to T184293: rack/setup/install lvs101[3-6].

Completed auto-reimage of hosts:

['lvs1014.eqiad.wmnet']
Sun, May 19, 12:00 AM · ops-eqiad, Operations, Traffic

Sat, May 18

ops-monitoring-bot added a comment to T184293: rack/setup/install lvs101[3-6].

Completed auto-reimage of hosts:

['lvs1013.eqiad.wmnet']
Sat, May 18, 11:59 PM · ops-eqiad, Operations, Traffic
ops-monitoring-bot added a comment to T184293: rack/setup/install lvs101[3-6].

Completed auto-reimage of hosts:

['lvs1015.eqiad.wmnet']
Sat, May 18, 11:20 PM · ops-eqiad, Operations, Traffic
ops-monitoring-bot added a comment to T184293: rack/setup/install lvs101[3-6].

Script wmf-auto-reimage was launched by bblack on cumin1001.eqiad.wmnet for hosts:

['lvs1015.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905182245_bblack_8750.log.

Sat, May 18, 10:45 PM · ops-eqiad, Operations, Traffic
ops-monitoring-bot added a comment to T184293: rack/setup/install lvs101[3-6].

Script wmf-auto-reimage was launched by bblack on cumin1001.eqiad.wmnet for hosts:

['lvs1013.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905182243_bblack_8547.log.

Sat, May 18, 10:44 PM · ops-eqiad, Operations, Traffic
ops-monitoring-bot added a comment to T184293: rack/setup/install lvs101[3-6].

Script wmf-auto-reimage was launched by bblack on cumin1001.eqiad.wmnet for hosts:

['lvs1014.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905182243_bblack_8549.log.

Sat, May 18, 10:44 PM · ops-eqiad, Operations, Traffic

Thu, May 16

ops-monitoring-bot added a comment to T223148: Cloud Services: reallocate workload from rack B5-eqiad.

Icinga downtime for 8:00:00 set by aborrero@cumin1001 on 1 host(s) and their services with reason: possible power outage

labweb1001.wikimedia.org
Thu, May 16, 11:00 AM · Patch-For-Review, cloud-services-team (Kanban), ops-eqiad, Operations
ops-monitoring-bot added a comment to T223148: Cloud Services: reallocate workload from rack B5-eqiad.

Icinga downtime for 8:00:00 set by aborrero@cumin1001 on 1 host(s) and their services with reason: possible power outage

cloudvirt1028.eqiad.wmnet
Thu, May 16, 11:00 AM · Patch-For-Review, cloud-services-team (Kanban), ops-eqiad, Operations
ops-monitoring-bot added a comment to T223148: Cloud Services: reallocate workload from rack B5-eqiad.

Icinga downtime for 8:00:00 set by aborrero@cumin1001 on 1 host(s) and their services with reason: possible power outage

cloudvirt1014.eqiad.wmnet
Thu, May 16, 11:00 AM · Patch-For-Review, cloud-services-team (Kanban), ops-eqiad, Operations

Tue, May 14

ops-monitoring-bot added projects to T223265: Degraded RAID on db1070: Operations, ops-eqiad.
Tue, May 14, 11:59 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223264: Degraded RAID on es2004: ops-codfw, Operations.
Tue, May 14, 11:58 AM · Operations, ops-codfw
ops-monitoring-bot added projects to T223258: Degraded RAID on labstore2002: ops-codfw, Operations.
Tue, May 14, 11:49 AM · Operations, ops-codfw
ops-monitoring-bot added projects to T223257: Degraded RAID on es2002: ops-codfw, Operations.
Tue, May 14, 11:49 AM · Operations, ops-codfw
ops-monitoring-bot added projects to T223256: Degraded RAID on labstore2001: ops-codfw, Operations.
Tue, May 14, 11:49 AM · Operations, ops-codfw
ops-monitoring-bot added projects to T223255: Degraded RAID on labstore2003: ops-codfw, Operations.
Tue, May 14, 11:49 AM · Operations, ops-codfw
ops-monitoring-bot added projects to T223254: Degraded RAID on db2096: ops-codfw, Operations.
Tue, May 14, 11:49 AM · Operations, ops-codfw
ops-monitoring-bot added projects to T223253: Degraded RAID on tungsten: Operations, ops-eqiad.
Tue, May 14, 11:49 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223252: Degraded RAID on db1072: Operations, ops-eqiad.
Tue, May 14, 11:49 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223251: Degraded RAID on ms-be1043: Operations, ops-eqiad.
Tue, May 14, 11:49 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223250: Degraded RAID on kafka1012: Operations, ops-eqiad.
Tue, May 14, 11:49 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223249: Degraded RAID on backup2001: ops-codfw, Operations.
Tue, May 14, 11:49 AM · Operations, ops-codfw
ops-monitoring-bot added projects to T223248: Degraded RAID on db1067: Operations, ops-eqiad.
Tue, May 14, 11:49 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223246: Degraded RAID on kafka1014: Operations, ops-eqiad.
Tue, May 14, 11:39 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223245: Degraded RAID on db1063: Operations, ops-eqiad.
Tue, May 14, 11:39 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223244: Degraded RAID on cloudvirt1018: Operations, ops-eqiad.
Tue, May 14, 11:39 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223243: Degraded RAID on db1066: Operations, ops-eqiad.
Tue, May 14, 11:37 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223242: Degraded RAID on db1071: Operations, ops-eqiad.
Tue, May 14, 11:37 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223241: Degraded RAID on cloudvirt1024: Operations, ops-eqiad.
Tue, May 14, 11:36 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223238: Degraded RAID on db1064: Operations, ops-eqiad.
Tue, May 14, 11:33 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223237: Degraded RAID on cloudvirt1015: Operations, ops-eqiad.
Tue, May 14, 11:33 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223236: Degraded RAID on dbstore1001: Operations, ops-eqiad.
Tue, May 14, 11:33 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223235: Degraded RAID on cloudstore1009: Operations, ops-eqiad.
Tue, May 14, 11:33 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223234: Degraded RAID on labstore1001: Operations, ops-eqiad.
Tue, May 14, 11:33 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223233: Degraded RAID on db1068: Operations, ops-eqiad.
Tue, May 14, 11:32 AM · ops-eqiad, Operations
ops-monitoring-bot added projects to T223232: Degraded RAID on labstore2004: ops-codfw, Operations.
Tue, May 14, 11:32 AM · Operations, ops-codfw
ops-monitoring-bot added a comment to T222937: Replace Varnish backends with ATS on cache upload nodes in esams.

Completed auto-reimage of hosts:

['cp3036.esams.wmnet']
Tue, May 14, 10:49 AM · Patch-For-Review, Operations, Traffic
ops-monitoring-bot added a comment to T222937: Replace Varnish backends with ATS on cache upload nodes in esams.

Script wmf-auto-reimage was launched by ema on cumin1001.eqiad.wmnet for hosts:

['cp3036.esams.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905141007_ema_128774.log.

Tue, May 14, 10:07 AM · Patch-For-Review, Operations, Traffic

Fri, May 10

ops-monitoring-bot added a comment to T222937: Replace Varnish backends with ATS on cache upload nodes in esams.

Completed auto-reimage of hosts:

['cp3038.esams.wmnet']
Fri, May 10, 1:10 PM · Patch-For-Review, Operations, Traffic
ops-monitoring-bot added a comment to T222937: Replace Varnish backends with ATS on cache upload nodes in esams.

Script wmf-auto-reimage was launched by ema on cumin1001.eqiad.wmnet for hosts:

['cp3038.esams.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905101228_ema_183031.log.

Fri, May 10, 12:28 PM · Patch-For-Review, Operations, Traffic
ops-monitoring-bot added a comment to T222772: Productionize db2[103-120].

Completed auto-reimage of hosts:

['db2114.codfw.wmnet']
Fri, May 10, 5:40 AM · Patch-For-Review, Goal, DBA
ops-monitoring-bot added a comment to T222772: Productionize db2[103-120].

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db2114.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905100455_marostegui_89703.log.

Fri, May 10, 4:55 AM · Patch-For-Review, Goal, DBA

Thu, May 9

ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Completed auto-reimage of hosts:

['db1133.eqiad.wmnet']
Thu, May 9, 10:22 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1133.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905090917_marostegui_103846.log.

Thu, May 9, 9:17 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations

Tue, May 7

ops-monitoring-bot added a comment to T219404: rack/setup/install restbase10[19-27].eqiad.wmnet.

Script wmf-auto-reimage was launched by robh on cumin1001.eqiad.wmnet for hosts:

['restbase1020.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905072356_robh_148203.log.

Tue, May 7, 11:56 PM · Cassandra, Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (Security, stability, performance and scalability (TEC1)), User-fgiunchedi, User-Eevans, ops-eqiad, Operations, RESTBase
ops-monitoring-bot added a comment to T219404: rack/setup/install restbase10[19-27].eqiad.wmnet.

Script wmf-auto-reimage was launched by robh on cumin1001.eqiad.wmnet for hosts:

['restbase1019.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905072339_robh_144413.log.

Tue, May 7, 11:39 PM · Cassandra, Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (Security, stability, performance and scalability (TEC1)), User-fgiunchedi, User-Eevans, ops-eqiad, Operations, RESTBase
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Completed auto-reimage of hosts:

['db1133.eqiad.wmnet']
Tue, May 7, 4:00 PM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T213263: Partial cache_upload traffic switchover to ATS and switchback to Varnish.

Completed auto-reimage of hosts:

['cp2009.codfw.wmnet']
Tue, May 7, 3:49 PM · Patch-For-Review, Operations, Traffic
ops-monitoring-bot added a comment to T213263: Partial cache_upload traffic switchover to ATS and switchback to Varnish.

Script wmf-auto-reimage was launched by ema on cumin1001.eqiad.wmnet for hosts:

['cp2009.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905071506_ema_20589.log.

Tue, May 7, 3:07 PM · Patch-For-Review, Operations, Traffic
ops-monitoring-bot added a comment to T213263: Partial cache_upload traffic switchover to ATS and switchback to Varnish.

Completed auto-reimage of hosts:

['cp1074.eqiad.wmnet', 'cp2021.codfw.wmnet', 'cp2015.codfw.wmnet', 'cp2003.codfw.wmnet']
Tue, May 7, 3:03 PM · Patch-For-Review, Operations, Traffic
ops-monitoring-bot added a comment to T213263: Partial cache_upload traffic switchover to ATS and switchback to Varnish.

Completed auto-reimage of hosts:

['cp1072.eqiad.wmnet', 'cp1073.eqiad.wmnet', 'cp1071.eqiad.wmnet']
Tue, May 7, 3:00 PM · Patch-For-Review, Operations, Traffic
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1133.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905071454_marostegui_16155.log.

Tue, May 7, 2:55 PM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T213263: Partial cache_upload traffic switchover to ATS and switchback to Varnish.

Script wmf-auto-reimage was launched by ema on cumin1001.eqiad.wmnet for hosts:

['cp1074.eqiad.wmnet', 'cp2003.codfw.wmnet', 'cp2015.codfw.wmnet', 'cp2021.codfw.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905071406_ema_260929.log.

Tue, May 7, 2:06 PM · Patch-For-Review, Operations, Traffic
ops-monitoring-bot added a comment to T213263: Partial cache_upload traffic switchover to ATS and switchback to Varnish.

Script wmf-auto-reimage was launched by ema on cumin1001.eqiad.wmnet for hosts:

['cp1071.eqiad.wmnet', 'cp1072.eqiad.wmnet', 'cp1073.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905071331_ema_248362.log.

Tue, May 7, 1:31 PM · Patch-For-Review, Operations, Traffic
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Completed auto-reimage of hosts:

['db1138.eqiad.wmnet', 'db1137.eqiad.wmnet']
Tue, May 7, 8:52 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1137.eqiad.wmnet', 'db1138.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905070836_marostegui_174999.log.

Tue, May 7, 8:36 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1133.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905070739_marostegui_163174.log.

Tue, May 7, 7:39 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Completed auto-reimage of hosts:

['db1136.eqiad.wmnet', 'db1134.eqiad.wmnet', 'db1135.eqiad.wmnet']
Tue, May 7, 7:36 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1134.eqiad.wmnet', 'db1135.eqiad.wmnet', 'db1136.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905070718_marostegui_155235.log.

Tue, May 7, 7:18 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Completed auto-reimage of hosts:

['db1131.eqiad.wmnet', 'db1132.eqiad.wmnet']
Tue, May 7, 7:07 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1131.eqiad.wmnet', 'db1132.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905070650_marostegui_146684.log.

Tue, May 7, 6:50 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Completed auto-reimage of hosts:

['db1132.eqiad.wmnet', 'db1133.eqiad.wmnet']
Tue, May 7, 6:47 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1131.eqiad.wmnet', 'db1132.eqiad.wmnet', 'db1133.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905070623_marostegui_139335.log.

Tue, May 7, 6:23 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Completed auto-reimage of hosts:

['db1127.eqiad.wmnet']
Tue, May 7, 6:21 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1127.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905070605_marostegui_134280.log.

Tue, May 7, 6:05 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Completed auto-reimage of hosts:

['db1130.eqiad.wmnet']
Tue, May 7, 6:04 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1130.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905070548_marostegui_129310.log.

Tue, May 7, 5:49 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Completed auto-reimage of hosts:

['db1130.eqiad.wmnet']
Tue, May 7, 5:46 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1128.eqiad.wmnet', 'db1129.eqiad.wmnet', 'db1130.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905070531_marostegui_123474.log.

Tue, May 7, 5:31 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Completed auto-reimage of hosts:

['db1126.eqiad.wmnet']
Tue, May 7, 5:27 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
ops-monitoring-bot added a comment to T211613: rack/setup/install db11[26-38].eqiad.wmnet.

Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts:

['db1126.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905070511_marostegui_118294.log.

Tue, May 7, 5:11 AM · Goal, DBA, ops-eqiad, User-Marostegui, Operations

Mon, May 6

ops-monitoring-bot added a comment to T219967: Replace Varnish backends with ATS on cache upload nodes in ulsfo .

Completed auto-reimage of hosts:

['cp4026.ulsfo.wmnet']
Mon, May 6, 3:08 PM · Patch-For-Review, Operations, Goal, Traffic
ops-monitoring-bot added projects to T222622: Degraded RAID on db2049: ops-codfw, Operations.
Mon, May 6, 2:54 PM · Operations, ops-codfw
ops-monitoring-bot added a comment to T219967: Replace Varnish backends with ATS on cache upload nodes in ulsfo .

Script wmf-auto-reimage was launched by ema on cumin2001.codfw.wmnet for hosts:

['cp4026.ulsfo.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905061425_ema_2856.log.

Mon, May 6, 2:25 PM · Patch-For-Review, Operations, Goal, Traffic

Sun, May 5

ops-monitoring-bot added projects to T222526: Degraded RAID on db2049: ops-codfw, Operations.
Sun, May 5, 4:02 AM · Operations, ops-codfw

Fri, May 3

ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Completed auto-reimage of hosts:

['cloudvirt1007.eqiad.wmnet']
Fri, May 3, 11:36 AM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)
ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudvirt1007.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/201905031040_aborrero_213590_cloudvirt1007_eqiad_wmnet.log.

Fri, May 3, 10:40 AM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)
ops-monitoring-bot added a comment to T219967: Replace Varnish backends with ATS on cache upload nodes in ulsfo .

Completed auto-reimage of hosts:

['cp4025.ulsfo.wmnet']
Fri, May 3, 10:37 AM · Patch-For-Review, Operations, Goal, Traffic
ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Completed auto-reimage of hosts:

['cloudvirt1007.eqiad.wmnet']
Fri, May 3, 10:34 AM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)
ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudvirt1007.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/201905031034_aborrero_211701_cloudvirt1007_eqiad_wmnet.log.

Fri, May 3, 10:34 AM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)
ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Completed auto-reimage of hosts:

['cloudvirt1007.eqiad.wmnet']
Fri, May 3, 10:33 AM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)
ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudvirt1007.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/201905031033_aborrero_211644_cloudvirt1007_eqiad_wmnet.log.

Fri, May 3, 10:33 AM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)
ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Completed auto-reimage of hosts:

['cloudvirt1007.eqiad.wmnet']
Fri, May 3, 10:31 AM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)
ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudvirt1007.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/201905031007_aborrero_206352_cloudvirt1007_eqiad_wmnet.log.

Fri, May 3, 10:11 AM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)
ops-monitoring-bot added a comment to T219967: Replace Varnish backends with ATS on cache upload nodes in ulsfo .

Script wmf-auto-reimage was launched by ema on cumin2001.codfw.wmnet for hosts:

['cp4025.ulsfo.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905030954_ema_30037.log.

Fri, May 3, 9:55 AM · Patch-For-Review, Operations, Goal, Traffic
ops-monitoring-bot added a comment to T219967: Replace Varnish backends with ATS on cache upload nodes in ulsfo .

Completed auto-reimage of hosts:

['cp4024.ulsfo.wmnet']
Fri, May 3, 8:33 AM · Patch-For-Review, Operations, Goal, Traffic
ops-monitoring-bot added a comment to T219967: Replace Varnish backends with ATS on cache upload nodes in ulsfo .

Script wmf-auto-reimage was launched by ema on cumin2001.codfw.wmnet for hosts:

['cp4024.ulsfo.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201905030750_ema_2659.log.

Fri, May 3, 7:50 AM · Patch-For-Review, Operations, Goal, Traffic

Thu, May 2

ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Completed auto-reimage of hosts:

['cloudvirt1007.eqiad.wmnet']
Thu, May 2, 5:18 PM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)
ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudvirt1007.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/201905021658_aborrero_255927_cloudvirt1007_eqiad_wmnet.log.

Thu, May 2, 4:58 PM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)
ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Completed auto-reimage of hosts:

['cloudvirt1007.eqiad.wmnet']
Thu, May 2, 4:54 PM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)
ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudvirt1007.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/201905021631_aborrero_250834_cloudvirt1007_eqiad_wmnet.log.

Thu, May 2, 4:31 PM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)
ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Completed auto-reimage of hosts:

['cloudvirt1007.eqiad.wmnet']
Thu, May 2, 4:16 PM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)
ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudvirt1007.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/201905021546_aborrero_241395_cloudvirt1007_eqiad_wmnet.log.

Thu, May 2, 3:46 PM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)
ops-monitoring-bot added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

Completed auto-reimage of hosts:

['cloudvirt1007.eqiad.wmnet']
Thu, May 2, 3:46 PM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)