Page MenuHomePhabricator
Create Task
Maniphest T242093

Upgrade cache cluster to debian buster
Closed, ResolvedPublic
Actions

Description

We are currently blocked on several tasks like rolling out TLS 1.3 till our caching cluster is upgraded to buster.

Upgrade status:

  • eqiad
    • text
      • cp1075
      • cp1077
      • cp1079
      • cp1081
      • cp1083
      • cp1085
      • cp1087
      • cp1089
    • upload
      • cp1076
      • cp1078
      • cp1080
      • cp1082
      • cp1084
      • cp1086
      • cp1088
      • cp1090
  • codfw
    • text
      • cp2001
      • cp2004
      • cp2006
      • cp2007
      • cp2010
      • cp2012
      • cp2013
      • cp2016
      • cp2019
      • cp2023
    • upload
      • cp2002
      • cp2005
      • cp2008
      • cp2011
      • cp2014
      • cp2017
      • cp2018
      • cp2020
      • cp2022
      • cp2024
      • cp2025
      • cp2026
  • esams
    • text
      • cp3050
      • cp3052
      • cp3054
      • cp3056
      • cp3058
      • cp3060
      • cp3062
      • cp3064
    • upload
      • cp3051
      • cp3053
      • cp3055
      • cp3057
      • cp3059
      • cp3061
      • cp3063
      • cp3065
  • ulsfo
    • text
      • cp4027
      • cp4028
      • cp4029
      • cp4030
      • cp4031
      • cp4032
    • upload
      • cp4021
      • cp4022
      • cp4023
      • cp4024
      • cp4025
      • cp4026
  • eqsin
    • text
      • cp5006
      • cp5007
      • cp5008
      • cp5009
      • cp5010
      • cp5011
      • cp5012
    • upload
      • cp5001
      • cp5002
      • cp5003
      • cp5004
      • cp5005
      • cp5006

Details

ProjectBranchLines +/-Subject
operations/puppetproduction+0 -16install_server: Reiamge {upload,text}@eqiad as buster
operations/puppetproduction+0 -26install_server: Reimage {upload,text}@codfw as buster
operations/puppetproduction+0 -15install_server: Reimage {upload,text}@esams as buster
operations/puppetproduction+0 -10install_server: Reimage {upload,text}@eqsin as buster
operations/puppetproduction+0 -5install_server: Reimage text@ulsfo as buster
operations/puppetproduction+0 -6install_server: Reimage upload@ulsfo as buster
operations/debs/trafficservermaster+23 -1Release 8.0.5-1wm14
operations/puppetproduction+0 -1install_server: Reimage cp3065 as buster
operations/puppetproduction+0 -1install_server: Reimage cp5012 as buster
operations/puppetproduction+0 -1install_server: Reimage cp5006 as buster
operations/puppetproduction+0 -1install_server: Reimage cp4032 as buster
operations/puppetproduction+3 -1install_server: Reimage cp4026 as buster
operations/puppetproduction+2 -1ATS: Move to a hiera parameter the ats-tls conftool service name
operations/puppetproduction+6 -35cache: Remove nginx from text and upload clusters
operations/puppetproduction+2 -0varnish: Set buster python version
operations/software/fifo-log-demuxmaster+6 -0Release 0.6.1
operations/software/varnish/vhtcpddebian+8 -1vhtcpd (0.1.2-2) buster-wikimedia; urgency=medium
operations/debs/trafficservermaster+16 -8Release 8.0.5-1wm13
operations/software/varnish/libvmod-tbfdebian+24 -13Release 2.0.91-2wm
operations/software/varnish/libvmod-netmapperdebian+11 -21.7-3: Rebuild for buster
operations/software/varnish/libvmod-re2debian+10 -21.3.1-3 Rebuild for buster
operations/software/varnish/varnishkafkadebian+14 -3varnishkafka (1.0.14-1) buster-wikimedia; urgency=medium
operations/debs/varnish4debian-wmf+10 -25.1.3-1wm12: Bump version and target buster
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Stashbot added a comment.Feb 12 2020, 3:02 PM

Mentioned in SAL (#wikimedia-operations) [2020-02-12T15:02:08Z] <vgutierrez> depool cp20[07,17] and reimage as buster - T242093

ops-monitoring-bot added a comment.Feb 12 2020, 3:03 PM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp2007.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002121502_vgutierrez_122677_cp2007_codfw_wmnet.log.

ops-monitoring-bot added a comment.Feb 12 2020, 3:12 PM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp2017.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002121512_vgutierrez_127278_cp2017_codfw_wmnet.log.

ops-monitoring-bot added a comment.Feb 12 2020, 3:21 PM

Completed auto-reimage of hosts:

['cp2007.codfw.wmnet']

Of which those FAILED:

['cp2007.codfw.wmnet']
ops-monitoring-bot added a comment.Feb 12 2020, 3:29 PM

Completed auto-reimage of hosts:

['cp2017.codfw.wmnet']

Of which those FAILED:

['cp2017.codfw.wmnet']
Stashbot added a comment.Feb 12 2020, 3:48 PM

Mentioned in SAL (#wikimedia-operations) [2020-02-12T15:48:33Z] <vgutierrez> pool cp20[07,17] running buster - T242093

Vgutierrez updated the task description. (Show Details)Feb 12 2020, 3:49 PM
Stashbot added a comment.Feb 12 2020, 3:50 PM

Mentioned in SAL (#wikimedia-operations) [2020-02-12T15:50:13Z] <vgutierrez> depool cp20[06,14] and reimage as buster - T242093

ops-monitoring-bot added a comment.Feb 12 2020, 3:50 PM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp2006.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002121550_vgutierrez_137927_cp2006_codfw_wmnet.log.

ops-monitoring-bot added a comment.Feb 12 2020, 3:53 PM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp2014.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002121553_vgutierrez_138370_cp2014_codfw_wmnet.log.

ops-monitoring-bot added a comment.Feb 12 2020, 4:12 PM

Completed auto-reimage of hosts:

['cp2006.codfw.wmnet']

and were ALL successful.

ops-monitoring-bot added a comment.Feb 12 2020, 4:14 PM

Completed auto-reimage of hosts:

['cp2014.codfw.wmnet']

and were ALL successful.

Stashbot added a comment.Feb 12 2020, 4:52 PM

Mentioned in SAL (#wikimedia-operations) [2020-02-12T16:52:14Z] <vgutierrez> pool cp20[06,14] running buster - T242093

Vgutierrez updated the task description. (Show Details)Feb 12 2020, 4:52 PM
Stashbot added a comment.Feb 13 2020, 5:09 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T05:09:44Z] <vgutierrez> depool cp20[04,11] and reimage as buster - T242093

ops-monitoring-bot added a comment.Feb 13 2020, 5:11 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp2004.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002130511_vgutierrez_23193_cp2004_codfw_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 5:13 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp2011.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002130513_vgutierrez_23547_cp2011_codfw_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 5:32 AM

Completed auto-reimage of hosts:

['cp2004.codfw.wmnet']

and were ALL successful.

Stashbot added a comment.Feb 13 2020, 5:34 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T05:34:57Z] <vgutierrez> pool cp2004 running buster - T242093

Stashbot added a comment.Feb 13 2020, 5:35 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T05:35:34Z] <vgutierrez> depool cp2001 and reimage as buster - T242093

ops-monitoring-bot added a comment.Feb 13 2020, 5:35 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp2001.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002130535_vgutierrez_29443_cp2001_codfw_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 5:35 AM

Completed auto-reimage of hosts:

['cp2011.codfw.wmnet']

and were ALL successful.

Stashbot added a comment.Feb 13 2020, 5:37 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T05:37:47Z] <vgutierrez> pool cp2011 running buster - T242093

Stashbot added a comment.Feb 13 2020, 5:38 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T05:38:24Z] <vgutierrez> depool cp2008 and reimage as buster - T242093

ops-monitoring-bot added a comment.Feb 13 2020, 5:39 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp2008.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002130538_vgutierrez_29767_cp2008_codfw_wmnet.log.

Vgutierrez updated the task description. (Show Details)Feb 13 2020, 5:39 AM
gerritbot added a comment.Feb 13 2020, 5:55 AM

Change 571862 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] install_server: Reiamge {upload,text}@eqiad as buster

https://gerrit.wikimedia.org/r/571862

gerritbot added a project: Patch-For-Review.Feb 13 2020, 5:55 AM
ops-monitoring-bot added a comment.Feb 13 2020, 5:58 AM

Completed auto-reimage of hosts:

['cp2001.codfw.wmnet']

and were ALL successful.

Stashbot added a comment.Feb 13 2020, 6:04 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T06:03:56Z] <vgutierrez> pool cp20[01,08] running buster - T242093

Stashbot added a comment.Feb 13 2020, 6:04 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T06:04:39Z] <vgutierrez> depool cp20[02,05] and reimage as buster - T242093

ops-monitoring-bot added a comment.Feb 13 2020, 6:05 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp2002.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002130605_vgutierrez_37262_cp2002_codfw_wmnet.log.

Vgutierrez updated the task description. (Show Details)Feb 13 2020, 6:05 AM
gerritbot added a comment.Feb 13 2020, 6:07 AM

Change 571862 merged by Vgutierrez:
[operations/puppet@production] install_server: Reiamge {upload,text}@eqiad as buster

https://gerrit.wikimedia.org/r/571862

ops-monitoring-bot added a comment.Feb 13 2020, 6:08 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp2005.codfw.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002130608_vgutierrez_37846_cp2005_codfw_wmnet.log.

Maintenance_bot removed a project: Patch-For-Review.Feb 13 2020, 6:10 AM
Stashbot added a comment.Feb 13 2020, 6:11 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T06:11:51Z] <vgutierrez> depool cp10[89,90] and reimage as buster - T242093

ops-monitoring-bot added a comment.Feb 13 2020, 6:12 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1089.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002130612_vgutierrez_38556_cp1089_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 6:16 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1090.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002130616_vgutierrez_39272_cp1090_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 6:23 AM

Completed auto-reimage of hosts:

['cp2002.codfw.wmnet']

Of which those FAILED:

['cp2002.codfw.wmnet']
ops-monitoring-bot added a comment.Feb 13 2020, 6:29 AM

Completed auto-reimage of hosts:

['cp2005.codfw.wmnet']

and were ALL successful.

ops-monitoring-bot added a comment.Feb 13 2020, 6:33 AM

Completed auto-reimage of hosts:

['cp1089.eqiad.wmnet']

Of which those FAILED:

['cp1089.eqiad.wmnet']
ops-monitoring-bot added a comment.Feb 13 2020, 6:37 AM

Completed auto-reimage of hosts:

['cp1090.eqiad.wmnet']

Of which those FAILED:

['cp1090.eqiad.wmnet']
Stashbot added a comment.Feb 13 2020, 6:49 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T06:49:39Z] <vgutierrez> pool cp20[02,05] running buster - T242093

Stashbot added a comment.Feb 13 2020, 7:02 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T07:01:59Z] <vgutierrez> pool cp10[89,90] running buster - T242093

Vgutierrez updated the task description. (Show Details)Feb 13 2020, 7:02 AM
Stashbot added a comment.Feb 13 2020, 7:03 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T07:03:12Z] <vgutierrez> depool cp10[87,88] and reimage as buster - T242093

ops-monitoring-bot added a comment.Feb 13 2020, 7:03 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1087.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002130703_vgutierrez_52539_cp1087_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 7:08 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1088.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002130708_vgutierrez_55107_cp1088_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 7:26 AM

Completed auto-reimage of hosts:

['cp1087.eqiad.wmnet']

Of which those FAILED:

['cp1087.eqiad.wmnet']
ops-monitoring-bot added a comment.Feb 13 2020, 7:29 AM

Completed auto-reimage of hosts:

['cp1088.eqiad.wmnet']

Of which those FAILED:

['cp1088.eqiad.wmnet']
Stashbot added a comment.Feb 13 2020, 7:49 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T07:49:50Z] <vgutierrez> pool cp10[87,88] running buster - T242093

Vgutierrez updated the task description. (Show Details)Feb 13 2020, 7:53 AM
Stashbot added a comment.Feb 13 2020, 7:57 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T07:57:08Z] <vgutierrez> depool cp10[85,86] and reimage as buster - T242093

ops-monitoring-bot added a comment.Feb 13 2020, 7:57 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1085.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002130757_vgutierrez_66095_cp1085_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 7:59 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1086.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002130757_vgutierrez_66140_cp1086_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 8:20 AM

Completed auto-reimage of hosts:

['cp1085.eqiad.wmnet']

Of which those FAILED:

['cp1085.eqiad.wmnet']
ops-monitoring-bot added a comment.Feb 13 2020, 8:27 AM

Completed auto-reimage of hosts:

['cp1086.eqiad.wmnet']

and were ALL successful.

Stashbot added a comment.Feb 13 2020, 9:45 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T09:45:06Z] <vgutierrez> pool cp10[85,86] running buster - T242093

Stashbot added a comment.Feb 13 2020, 9:45 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T09:45:54Z] <vgutierrez> depool cp10[83,84] and reimage as buster - T242093

ops-monitoring-bot added a comment.Feb 13 2020, 9:46 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1083.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002130946_vgutierrez_88180_cp1083_eqiad_wmnet.log.

Vgutierrez updated the task description. (Show Details)Feb 13 2020, 9:46 AM
ops-monitoring-bot added a comment.Feb 13 2020, 9:48 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1084.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002130948_vgutierrez_88590_cp1084_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 10:11 AM

Completed auto-reimage of hosts:

['cp1084.eqiad.wmnet']

Of which those FAILED:

['cp1084.eqiad.wmnet']
ops-monitoring-bot added a comment.Feb 13 2020, 10:14 AM

Completed auto-reimage of hosts:

['cp1083.eqiad.wmnet']

and were ALL successful.

Stashbot added a comment.Feb 13 2020, 10:21 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T10:21:34Z] <vgutierrez> pool cp10[83,84] running buster - T242093

Stashbot added a comment.Feb 13 2020, 10:24 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T10:24:13Z] <vgutierrez> depool cp10[81,82] and reimage as buster - T242093

ops-monitoring-bot added a comment.Feb 13 2020, 10:24 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1081.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002131024_vgutierrez_98289_cp1081_eqiad_wmnet.log.

Vgutierrez updated the task description. (Show Details)Feb 13 2020, 10:24 AM
ops-monitoring-bot added a comment.Feb 13 2020, 10:26 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1082.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002131026_vgutierrez_98684_cp1082_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 10:40 AM

Completed auto-reimage of hosts:

['cp1081.eqiad.wmnet']

Of which those FAILED:

['cp1081.eqiad.wmnet']
ops-monitoring-bot added a comment.Feb 13 2020, 10:42 AM

Completed auto-reimage of hosts:

['cp1082.eqiad.wmnet']

Of which those FAILED:

['cp1082.eqiad.wmnet']
Stashbot added a comment.Feb 13 2020, 11:02 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T11:02:53Z] <vgutierrez> pool cp10[81,82] and reimage as buster - T242093

Vgutierrez updated the task description. (Show Details)Feb 13 2020, 11:06 AM
Stashbot added a comment.Feb 13 2020, 11:16 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T11:16:37Z] <vgutierrez> depool cp10[79,80] and reimage as buster - T242093

ops-monitoring-bot added a comment.Feb 13 2020, 11:17 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1079.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002131117_vgutierrez_109648_cp1079_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 11:20 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1080.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002131119_vgutierrez_110041_cp1080_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 11:46 AM

Completed auto-reimage of hosts:

['cp1079.eqiad.wmnet']

and were ALL successful.

ops-monitoring-bot added a comment.Feb 13 2020, 11:48 AM

Completed auto-reimage of hosts:

['cp1080.eqiad.wmnet']

and were ALL successful.

Stashbot added a comment.Feb 13 2020, 11:52 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T11:52:13Z] <vgutierrez> pool cp10[79,80] running buster - T242093

Vgutierrez updated the task description. (Show Details)Feb 13 2020, 11:53 AM
Stashbot added a comment.Feb 13 2020, 11:53 AM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T11:53:43Z] <vgutierrez> depool cp10[77,78] and reimage as buster - T242093

ops-monitoring-bot added a comment.Feb 13 2020, 11:54 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1077.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002131154_vgutierrez_118940_cp1077_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 11:56 AM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1078.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002131156_vgutierrez_119341_cp1078_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 12:23 PM

Completed auto-reimage of hosts:

['cp1077.eqiad.wmnet']

and were ALL successful.

ops-monitoring-bot added a comment.Feb 13 2020, 12:25 PM

Completed auto-reimage of hosts:

['cp1078.eqiad.wmnet']

and were ALL successful.

Stashbot added a comment.Feb 13 2020, 12:28 PM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T12:28:14Z] <vgutierrez> pool cp10[77,78] running buster - T242093

Vgutierrez updated the task description. (Show Details)Feb 13 2020, 12:28 PM
Stashbot added a comment.Feb 13 2020, 12:29 PM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T12:29:00Z] <vgutierrez> depool cp10[75,76] and reimage as buster - T242093

ops-monitoring-bot added a comment.Feb 13 2020, 12:29 PM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1075.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002131229_vgutierrez_127126_cp1075_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 12:31 PM

Script wmf-auto-reimage was launched by vgutierrez on cumin1001.eqiad.wmnet for hosts:

cp1076.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202002131231_vgutierrez_127380_cp1076_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Feb 13 2020, 12:58 PM

Completed auto-reimage of hosts:

['cp1075.eqiad.wmnet']

and were ALL successful.

ops-monitoring-bot added a comment.Feb 13 2020, 1:00 PM

Completed auto-reimage of hosts:

['cp1076.eqiad.wmnet']

and were ALL successful.

Stashbot added a comment.Feb 13 2020, 1:00 PM

Mentioned in SAL (#wikimedia-operations) [2020-02-13T13:00:51Z] <vgutierrez> pool cp10[75,76] running buster - T242093

Vgutierrez closed this task as Resolved.Feb 13 2020, 1:01 PM
Vgutierrez claimed this task.
Vgutierrez updated the task description. (Show Details)
vgutierrez@cumin1001:~$ sudo -i cumin 'A:cp' 'cat /etc/debian_version'
78 hosts will be targeted:
cp[2001-2002,2004-2008,2010-2014,2016-2020,2022-2026].codfw.wmnet,cp[1075-1090].eqiad.wmnet,cp[5001-5012].eqsin.wmnet,cp[3050-3065].esams.wmnet,cp[4021-4032].ulsfo.wmnet
Confirm to continue [y/n]? y
===== NODE GROUP =====
(78) cp[2001-2002,2004-2008,2010-2014,2016-2020,2022-2026].codfw.wmnet,cp[1075-1090].eqiad.wmnet,cp[5001-5012].eqsin.wmnet,cp[3050-3065].esams.wmnet,cp[4021-4032].ulsfo.wmnet
----- OUTPUT of 'cat /etc/debian_version' -----
10.3
Krinkle mentioned this in T247020: load.php?modules=startup miss rate tripled on 2020-02-05.Mar 5 2020, 7:29 PM
Vgutierrez mentioned this in T238305: Servers freezing across the caching cluster.Apr 2 2020, 5:37 AM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL