Page MenuHomePhabricator

replace all Ubuntu (trusty) hosts in production with Debian
Closed, ResolvedPublic

Description

This is a ticket to track the remaining trusty (now Ubuntu-in-general) hosts in production and upgrade them by reinstalling with Debian (preferably stable, oldstable (jessie) if really needed).

It should be resolved once we have completed the switch to Debian-only production environment.

It is like the tickets in the past for lucid (T80945) and precise (T123525) but for trusty now.

It is closely related to T186029 which is about removing trusty support in cloud VPS.

"In April 2019 the Ubuntu project will stop providing security patch support for the Ubuntu 14.04 "Trusty" release."

currently left: '67'

[neodymium:~] $ sudo cumin 'F:lsbdistid = Ubuntu'
70 hosts will be targeted:
californium.wikimedia.org,db[1009,1011,1020,1030-1031,1034,1039].eqiad.wmnet,
dbstore1002.eqiad.wmnet,eventlog1001.eqiad.wmnet,francium.eqiad.wmnet,iridium.eqiad.wmnet,
labcontrol[1001-1004].wikimedia.org,labnet[1001-1002].eqiad.wmnet,labsdb[1001,1003].eqiad.wmnet,
labservices[1001-1002].wikimedia.org,labstore1003.eqiad.wmnet,labtestcontrol[2001,2003].wikimedia.org,
labtestmetal2001.codfw.wmnet,labtestnet[2001-2002].codfw.wmnet,labtestneutron[2001-2002].codfw.wmnet,
labtestservices[2001-2003].wikimedia.org,labtestvirt[2001-2003].codfw.wmnet,labtestweb2001.wikimedia.org,
labvirt[1001-1020].eqiad.wmnet,ocg[1001-1003].eqiad.wmnet,sca[2003-2004].codfw.wmnet,
sca[1003-1004].eqiad.wmnet,silver.wikimedia.org,snapshot[1001,1005-1007].eqiad.wmnet,stat1003.eqiad.wmnet
  • californium.wikimedia.org T189921
  • db[1009,1011,1020,1030-1031,1034,1039].eqiad.wmnet
  • dbstore1002.eqiad.wmnet
  • eventlog1001.eqiad.wmnet - migrated, awaiting decom in T189566
  • francium.eqiad.wmnet
  • iridium.eqiad.wmnet - decommissioned (powered off) on T172487 (T152129)
  • labcontrol[1001-1004].wikimedia.org
  • labnet[1001-1002].eqiad.wmnet
  • labsdb[1001,1003].eqiad.wmnet
  • labservices[1001-1002].wikimedia.org
  • labstore1003.eqiad.wmnet
  • labtestcontrol[2001,2003].wikimedia.org
  • labtestmetal2001.codfw.wmnet
  • labtestnet[2001-2002].codfw.wmnet
  • labtestneutron[2001-2002].codfw.wmnet,
  • labtestservices[2001-2003].wikimedia.org,
  • labtestvirt[2001-2002].codfw.wmnet
  • labtestvirt2003.codfw.wmnet
  • labtestweb2001.wikimedia.org,
  • labvirt[1001-1020].eqiad.wmnet
  • ocg[1001-1003].eqiad.wmnet
  • sca[2003-2004].codfw.wmnet
  • sca[1003-1004].eqiad.wmnet
  • silver.wikimedia.org – pending migration, then decom: T168559
  • snapshot[1001,1005-1007].eqiad.wmnet -- awaiting decom: T181029
  • stat1003.eqiad.wmnet – migrated, awaiting decom: T175150

Event Timeline

Dzahn renamed this task from upgrade all Ubuntu (trusty) hosts in production to replace all Ubuntu (trusty) hosts in production with Debian.Feb 2 2018, 1:18 AM
Dzahn added a project: cloud-services-team.
RobH triaged this task as High priority.Feb 8 2018, 7:06 PM
RobH subscribed.

Changing from 'needs triage' to 'high' priority, since this is tied to quarterly goals.

Krinkle updated the task description. (Show Details)

Changing from 'needs triage' to 'high' priority, since this is tied to quarterly goals.

Who's goal and which quarter? I can guarantee you that the labs* hosts will not be converted in FY17/18 Q3 and probably a large number of them will not be converted until sometime in FY18/19.

I thought it was an ongoing goal to replace them, perhaps its not perhaps a goal, but an ongoing project that has been gaining priority as its dragged on. I'm sure we'll be able to find out in sre weekly meeting.

If you think this shouldn't be high, but normal, that is also fine, but I didn't think it should sit with needs triage, since it wasn't something thats unbreak now and is either a high or normal priority as far as I can tell.

chasemp updated the task description. (Show Details)
chasemp updated the task description. (Show Details)

Script wmf-auto-reimage was launched by ariel on neodymium.eqiad.wmnet for hosts:

francium.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/201806030605_ariel_3116_francium_eqiad_wmnet.log.

Completed auto-reimage of hosts:

['francium.eqiad.wmnet']

and were ALL successful.

Since the tasks are marked as done, I think these servers are able to be checked off.

whoops, not decommissioned yet