Page MenuHomePhabricator

replace all Ubuntu (trusty) hosts in production with Debian
Closed, ResolvedPublic

Description

This is a ticket to track the remaining trusty (now Ubuntu-in-general) hosts in production and upgrade them by reinstalling with Debian (preferably stable, oldstable (jessie) if really needed).

It should be resolved once we have completed the switch to Debian-only production environment.

It is like the tickets in the past for lucid (T80945) and precise (T123525) but for trusty now.

It is closely related to T186029 which is about removing trusty support in cloud VPS.

"In April 2019 the Ubuntu project will stop providing security patch support for the Ubuntu 14.04 "Trusty" release."

currently left: '67'

[neodymium:~] $ sudo cumin 'F:lsbdistid = Ubuntu'
70 hosts will be targeted:
californium.wikimedia.org,db[1009,1011,1020,1030-1031,1034,1039].eqiad.wmnet,
dbstore1002.eqiad.wmnet,eventlog1001.eqiad.wmnet,francium.eqiad.wmnet,iridium.eqiad.wmnet,
labcontrol[1001-1004].wikimedia.org,labnet[1001-1002].eqiad.wmnet,labsdb[1001,1003].eqiad.wmnet,
labservices[1001-1002].wikimedia.org,labstore1003.eqiad.wmnet,labtestcontrol[2001,2003].wikimedia.org,
labtestmetal2001.codfw.wmnet,labtestnet[2001-2002].codfw.wmnet,labtestneutron[2001-2002].codfw.wmnet,
labtestservices[2001-2003].wikimedia.org,labtestvirt[2001-2003].codfw.wmnet,labtestweb2001.wikimedia.org,
labvirt[1001-1020].eqiad.wmnet,ocg[1001-1003].eqiad.wmnet,sca[2003-2004].codfw.wmnet,
sca[1003-1004].eqiad.wmnet,silver.wikimedia.org,snapshot[1001,1005-1007].eqiad.wmnet,stat1003.eqiad.wmnet
  • californium.wikimedia.org T189921
  • db[1009,1011,1020,1030-1031,1034,1039].eqiad.wmnet
  • dbstore1002.eqiad.wmnet
  • eventlog1001.eqiad.wmnet - migrated, awaiting decom in T189566
  • francium.eqiad.wmnet
  • iridium.eqiad.wmnet - decommissioned (powered off) on T172487 (T152129)
  • labcontrol[1001-1004].wikimedia.org
  • labnet[1001-1002].eqiad.wmnet
  • labsdb[1001,1003].eqiad.wmnet
  • labservices[1001-1002].wikimedia.org
  • labstore1003.eqiad.wmnet
  • labtestcontrol[2001,2003].wikimedia.org
  • labtestmetal2001.codfw.wmnet
  • labtestnet[2001-2002].codfw.wmnet
  • labtestneutron[2001-2002].codfw.wmnet,
  • labtestservices[2001-2003].wikimedia.org,
  • labtestvirt[2001-2002].codfw.wmnet
  • labtestvirt2003.codfw.wmnet
  • labtestweb2001.wikimedia.org,
  • labvirt[1001-1020].eqiad.wmnet
  • ocg[1001-1003].eqiad.wmnet
  • sca[2003-2004].codfw.wmnet
  • sca[1003-1004].eqiad.wmnet
  • silver.wikimedia.org – pending migration, then decom: T168559
  • snapshot[1001,1005-1007].eqiad.wmnet -- awaiting decom: T181029
  • stat1003.eqiad.wmnet – migrated, awaiting decom: T175150

Event Timeline

Dzahn created this task.Feb 2 2018, 1:13 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 2 2018, 1:13 AM
Dzahn updated the task description. (Show Details)Feb 2 2018, 1:17 AM
Paladox added a subscriber: Paladox.Feb 2 2018, 1:18 AM
Dzahn renamed this task from upgrade all Ubuntu (trusty) hosts in production to replace all Ubuntu (trusty) hosts in production with Debian.Feb 2 2018, 1:18 AM
Dzahn added a project: cloud-services-team.
Dzahn updated the task description. (Show Details)Feb 2 2018, 1:23 AM
bd808 updated the task description. (Show Details)Feb 2 2018, 1:42 AM
RobH triaged this task as High priority.Feb 8 2018, 7:06 PM
RobH added a subscriber: RobH.

Changing from 'needs triage' to 'high' priority, since this is tied to quarterly goals.

Dzahn updated the task description. (Show Details)Feb 8 2018, 8:40 PM
Krinkle updated the task description. (Show Details)Feb 8 2018, 8:43 PM
Krinkle updated the task description. (Show Details)
RobH updated the task description. (Show Details)Feb 8 2018, 8:51 PM
Dzahn updated the task description. (Show Details)Feb 8 2018, 9:20 PM
Dzahn updated the task description. (Show Details)Feb 8 2018, 9:26 PM
Dzahn updated the task description. (Show Details)Feb 8 2018, 10:00 PM
bd808 added a subscriber: bd808.Feb 8 2018, 11:40 PM

Changing from 'needs triage' to 'high' priority, since this is tied to quarterly goals.

Who's goal and which quarter? I can guarantee you that the labs* hosts will not be converted in FY17/18 Q3 and probably a large number of them will not be converted until sometime in FY18/19.

RobH added a comment.EditedFeb 8 2018, 11:44 PM

I thought it was an ongoing goal to replace them, perhaps its not perhaps a goal, but an ongoing project that has been gaining priority as its dragged on. I'm sure we'll be able to find out in sre weekly meeting.

If you think this shouldn't be high, but normal, that is also fine, but I didn't think it should sit with needs triage, since it wasn't something thats unbreak now and is either a high or normal priority as far as I can tell.

chasemp updated the task description. (Show Details)
chasemp updated the task description. (Show Details)
ArielGlenn updated the task description. (Show Details)Jun 3 2018, 6:04 AM

Script wmf-auto-reimage was launched by ariel on neodymium.eqiad.wmnet for hosts:

francium.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/201806030605_ariel_3116_francium_eqiad_wmnet.log.

Completed auto-reimage of hosts:

['francium.eqiad.wmnet']

and were ALL successful.

ArielGlenn updated the task description. (Show Details)Jun 3 2018, 7:26 AM
ArielGlenn updated the task description. (Show Details)
elukey updated the task description. (Show Details)Jun 3 2018, 7:38 AM
TerraCodes updated the task description. (Show Details)Jun 3 2018, 9:03 AM

Since the tasks are marked as done, I think these servers are able to be checked off.

TerraCodes updated the task description. (Show Details)Jun 3 2018, 9:57 AM

whoops, not decommissioned yet

MoritzMuehlenhoff closed this task as Resolved.Apr 11 2019, 7:13 AM
MoritzMuehlenhoff claimed this task.

This ticket is superseded by https://phabricator.wikimedia.org/T212772