Page MenuHomePhabricator

Track remaining jessie systems in production
Closed, ResolvedPublic

Description

This meta tasks tracks the migration of the remaining servers running Debian jessie towards either Stretch or Buster.

also see: T247045 (Migrate all of production metal to Buster or later)

  • cloudcontrol1003.wikimedia.org T221770
  • cloudcontrol1004.wikimedia.org T221770
  • cloudservices1003.wikimedia.org T221769
  • cloudservices1004.wikimedia.org T221769
  • cloudvirtan1001.eqiad.wmnet T224566
  • cloudvirtan1002.eqiad.wmnet T224566
  • cloudvirtan1003.eqiad.wmnet T224566
  • cloudvirtan1004.eqiad.wmnet T224566
  • cloudvirtan1005.eqiad.wmnet T224566
  • conf2001.codfw.wmnet T224560 (migrated to conf2004/2005/2006)
  • conf2002.codfw.wmnet T224560 (migrated to conf2004/2005/2006)
  • conf2003.codfw.wmnet T224560 (migrated to conf2004/2005/2006)
  • dbmonitor1001.wikimedia.org T224589
  • dbmonitor2001.wikimedia.org T224589
  • dubnium.wikimedia.org T224557 (replaced by ldap-corp1001)
  • pollux.wikimedia.org T224557 (replaced by ldap-corp2001)
  • poolcounter2001.codfw.wmnet T224572
  • poolcounter2002.codfw.wmnet T224572
  • poolcounter1001.eqiad.wmnet T224572
  • poolcounter1003.eqiad.wmnet T224572
  • dumpsdata[1001-1002].eqiad.wmnet T224563
  • kubestagetcd1001.eqiad.wmnet T224568 (replaced by new Buster hosts)
  • kubestagetcd1002.eqiad.wmnet T224568 (replaced by new Buster hosts)
  • kubestagetcd1003.eqiad.wmnet T224568 (replaced by new Buster hosts)
  • kubetcd2001.codfw.wmnet T239835 (replaced by kubetcd2004)
  • kubetcd2002.codfw.wmnet T239835 (replaced by kubetcd2005)
  • kubetcd2003.codfw.wmnet T239835 (replaced by kubetcd2006)
  • labmon1001.eqiad.wmnet T224585 (replaced by cloudmetrics1001)
  • labmon1002.eqiad.wmnet T224585 (replaced by cloudmetrics1002)
  • labpuppetmaster.1001.wikimedia.org
  • labpuppetmaster.1002.wikimedia.org
  • labtestpuppetmaster2001.wikimedia.org -> The WMCS puppet masters are being migrated into Cloud VPS T171188
  • labstore2001.codfw.wmnet -> Will be discarded once cloudbackup200[1,2].wikimedia.org are setup, see T214835
  • labstore2002.codfw.wmnet -> Will be discarded once cloudbackup200[1,2].wikimedia.org are setup, see T214835
  • labstore2003.codfw.wmnet -> Will be discarded once cloudbackup200[1,2].wikimedia.org are setup, see T214835
  • labstore2004.codfw.wmnet -> Will be discarded once cloudbackup200[1,2].wikimedia.org are setup, see T214835
  • labstore1006.wikimedia.org T224583
  • labstore1007.wikimedia.org T224583
  • wezen.codfw.wmnet (now centrallog2001.codfw.wmnet) T224564
  • mwlog1001.eqiad.wmnet T224565 (migrated to mwlog1002)
  • mwlog2001.codfw.wmnet T224565 (migrated to mwlog2002)
  • netmon1003.wikimedia.org (decomissioned in T198939)
  • pybal-test2001.codfw.wmnet T224570
  • restbase[2009-2012].codfw.wmnet T224553
  • restbase[1017-1018].eqiad.wmnet T224553
  • restbase-dev1004.eqiad.wmnet T224554
  • restbase-dev1005.eqiad.wmnet T224554
  • restbase-dev1006.eqiad.wmnet T224554
  • scb[1001-1004].eqiad.wmnet -> Remaining services are being migrated to Kubernetes, then it's obsolete
  • scb[2001-2006].codfw.wmnet -> Remaining services are being migrated to Kubernetes, then it's obsolete
  • tungsten.eqiad.wmnet -> This is blocked on migrating xhgui to the webperf hosts T180761, now using new VMs instead (T238098), role needs to support buster: T238788
  • ununpentium.wikimedia.org T180641

Hosts pending decommission:

Decommissioned hosts:

  • auth1001.eqiad.wmnet
  • bast3002.wikimedia.org
  • eeden.wikimedia.org
  • kafka1012.eqiad.wmnet
  • kafka1013.eqiad.wmnet
  • kafka1014.eqiad.wmnet
  • kafka1020.eqiad.wmnet
  • kafka1022.eqiad.wmnet
  • kafka1023.eqiad.wmnet
  • restbase[1007-1015].eqiad.wmnet T226715
  • labstore1001.eqiad.wmnet T187456
  • labstore1002.eqiad.wmnet T187456
  • darmstadtium.eqiad.wmnet T224562
  • lithium.eqiad.wmnet T229557
  • iron.wikimedia.org T220505
  • neodymium.eqiad.wmnet T220503
  • sarin.codfw.wmnet T220504
  • etherpad1001.eqiad.wmnet T224580

Related Objects

StatusSubtypeAssignedTask
ResolvedMoritzMuehlenhoff
ResolvedMoritzMuehlenhoff
Resolvedjijiki
Resolvedelukey
Resolvedelukey
Resolvedjbond
Resolvedjijiki
Resolvedjijiki
Resolvedhashar
Resolved mobrovac
ResolvedMoritzMuehlenhoff
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
DuplicateDzahn
DeclinedDzahn
ResolvedMoritzMuehlenhoff
ResolvedMoritzMuehlenhoff
ResolvedJMeybohm
DuplicateNone
Resolved fsero
ResolvedArielGlenn
ResolvedArielGlenn
ResolvedArielGlenn
Resolvedfgiunchedi
ResolvedPapaul
Resolvedherron
Resolvedtaavi
Resolvedhashar
Resolved Cmjohnson
ResolvedPapaul
ResolvedAndrew
ResolvedMoritzMuehlenhoff
ResolvedRequestMoritzMuehlenhoff
ResolvedRequestMoritzMuehlenhoff
DuplicateNone
Resolvedakosiaris
ResolvedVgutierrez
ResolvedMoritzMuehlenhoff
ResolvedMoritzMuehlenhoff
DuplicateNone
DuplicateDzahn
ResolvedDzahn
DuplicateRequestDzahn
Resolved Cmjohnson
ResolvedDzahn
ResolvedDzahn
InvalidRequestNone
InvalidRequestNone
DuplicateNone
ResolvedMoritzMuehlenhoff
Resolvedtaavi
ResolvedDzahn
ResolvedDzahn
Resolved Bstorm
Resolved Bstorm
Resolved Bstorm
Resolved Phamhi
ResolvedAndrew
Resolvedherron
ResolvedMoritzMuehlenhoff
Resolved Marostegui
Resolvedakosiaris
ResolvedDzahn
Resolvedhashar
DeclinedMoritzMuehlenhoff
Invalidthcipriani
Resolved mmodell
Resolvedhashar
ResolvedJoe
ResolvedJMeybohm
ResolvedJMeybohm
Resolvedaborrero
Resolvedaborrero
Resolvedaborrero
ResolvedPapaul
Resolved JHedden
Resolvedaborrero
Resolvedaborrero
ResolvedPapaul
Resolvedaborrero
Resolvedaborrero
Resolvedaborrero
Resolvedaborrero
ResolvedAndrew
Resolvedaborrero
Resolvedaborrero
ResolvedAndrew
Resolvedaborrero
Resolvedaborrero
ResolvedAndrew
Resolved Marostegui
Resolvedaborrero
ResolvedAndrew
DuplicateNone
ResolvedAndrew
ResolvedAndrew
Invalid JHedden
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
Resolvedjcrespo

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Dzahn updated the task description. (Show Details)
Dzahn updated the task description. (Show Details)

percentage of jessie systems left as of today (because we were asked): 4.2%

I have proposed to do the final steps for decommission of old backups servers heze and helium in FY2021Q3. T260717 (CC @akosiaris @MoritzMuehlenhoff - I don't think I will require anything specific from you except maybe the occasional review to cleanup old puppet code, but giving advance heads up in case it helps).

the occasional review to cleanup old puppet code

there is only https://gerrit.wikimedia.org/r/c/operations/puppet/+/626460/ DHCP and an entry check-microcode.py: blacklist_mds = ['helium']. That's it.

I have started the decommissioning process oh helium and heze and marked the tracking tasks on the description. It may take a bit to process them as we need to do lots of cleanup (beyond the previously proposed patch), but work has started.

helium and heze are now in hands of the respective dc-ops owners (tracked on the tickets on the description) and software-wise (puppet/traffic), removed from our production.

the occasional review to cleanup old puppet code

there is only https://gerrit.wikimedia.org/r/c/operations/puppet/+/626460/ DHCP and an entry check-microcode.py: blacklist_mds = ['helium']. That's it.

All of this was done at: https://gerrit.wikimedia.org/r/c/operations/puppet/+/658969

Revert of "Downgrade to TLSv1 for backup" is now tracked at T273182, only conf2* (jessie) hosts are relevant for that.

Change 689686 had a related patch set uploaded (by Volans; author: Volans):

[operations/puppet@production] cumin: remove jessie from distro aliases

https://gerrit.wikimedia.org/r/689686

Change 689686 merged by Volans:

[operations/puppet@production] cumin: remove jessie from distro aliases

https://gerrit.wikimedia.org/r/689686

MoritzMuehlenhoff claimed this task.
MoritzMuehlenhoff updated the task description. (Show Details)

This is completed \o/