Page MenuHomePhabricator
Create Task
Maniphest T247045

Migrate all of production metal and VMs to Buster or later
Open, LowPublic
Actions

Description

< T168494 (jessie->stretch) | T291916 (buster->bullseye) >

Top-level epic task to see progress.

Related Objects
Search...

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.
StatusSubtypeAssignedTask
· · ·
StalledNoneT302086 Set scap minimum python version to 3.7
OpenNoneT247045 Migrate all of production metal and VMs to Buster or later
ResolvedMoritzMuehlenhoffT245114 Migrate Cumin hosts to Buster
ResolvedRKemperT244753 Migrate WDQS to Debian Buster
DeclinedNoneT244736 Migrate Elasticsearch to Debian Buster
ResolvedbkingT289135 Upgrade Cirrus Elasticsearch clusters to Debian Bullseye
ResolvedakosiarisT239835 setup new, buster based, kubernetes etcd servers for staging/codfw/eqiad cluster
ResolvedDzahnT247648 miscweb1001/2001 - upgrade to buster or decom
ResolvedDzahnT247653 replace doc1001.eqiad.wmnet with a buster VM and create the codfw equivalent
ResolvedDzahnT247652 replace backends for releases.wikimedia.org with buster VMs
ResolvedDzahnT247651 upgrade planet.wikimedia.org backends to buster
ResolvedDzahnT247650 replace bromine and vega with buster VMs
ResolvedDzahnT247649 upgrade people.wikimedia.org backend to buster
ResolvedBstormT224583 Migrate labstore1006/1007 to Stretch/Buster
OpenNoneT216815 Upgrade Thumbor to Buster
Resolved• razziT231067 Install Debian Buster on Hadoop
ResolvedjijikiT213089 Upgrade memcached cluster to Debian Buster
ResolvedjijikiT265643 Upgrade MediaWiki's Redis cluster to Debian Buster
ResolvedDzahnT224576 Upgrade install servers to Buster
ResolvedakosiarisT245272 Draft a plan for upgrading kubernetes machines to buster
Restricted Task
ResolvedDzahnT245757 Upgrade MediaWiki clusters to Debian Buster (debian 10)
ResolvedMarosteguiT246604 Install 1 buster+10.4 host per section
ResolvedfgiunchediT247962 Migrate role::prometheus to Bullseye
ResolvedfgiunchediT247963 Migrate role::graphite::production to Bullseye
ResolvedlmataT247966 Migrate role::alerting_host to Buster
ResolvedfgiunchediT247967 Migrate role::netmon to Buster
ResolvedfgiunchediT247968 Migrate logging::webrequest::ops to Buster
ResolvedhasharT224591 Migrate contint* hosts to Buster
ResolvedjcrespoT236406 Switchover backup director service from helium to backup1001
ResolvedakosiarisT249724 Track and remove jessie based container images from production
ResolvedMarosteguiT250666 Upgrade WMF database-and-backup-related hosts to buster
ResolvedMoritzMuehlenhoffT224589 Migrate dbmonitor hosts to Buster
ResolvedMoritzMuehlenhoffT253779 Migrate role::bastionhost::general and role::bastionhost::pop to Buster
ResolvedhnowlanT264292 Migrate maps to Buster
ResolvedLegoktmT272550 Upgrade docker-registry servers to Debian Buster
ResolvedhnowlanT274119 Copy cassandra packages to buster-wikimedia
InvalidNoneT312595 ms-be2028 on stretch
· · ·

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 6 2020, 12:35 AM
Jdforrester-WMF renamed this task from Migrate all of production to Buster or later to Migrate all of production metal to Buster or later.Mar 6 2020, 12:35 AM
Jdforrester-WMF added subtasks: T245114: Migrate Cumin hosts to Buster, T244753: Migrate WDQS to Debian Buster, T244736: Migrate Elasticsearch to Debian Buster.
Jdforrester-WMF added a subtask: T239835: setup new, buster based, kubernetes etcd servers for staging/codfw/eqiad cluster.Mar 6 2020, 12:39 AM
akosiaris triaged this task as Low priority.Mar 6 2020, 11:30 AM
Dzahn added a subscriber: Dzahn.Mar 14 2020, 12:08 AM
Dzahn added subtasks: T247648: miscweb1001/2001 - upgrade to buster or decom, T247653: replace doc1001.eqiad.wmnet with a buster VM and create the codfw equivalent, T247652: replace backends for releases.wikimedia.org with buster VMs, T247651: upgrade planet.wikimedia.org backends to buster, T247650: replace bromine and vega with buster VMs, T247649: upgrade people.wikimedia.org backend to buster.Mar 14 2020, 12:10 AM
Dzahn added a subscriber: Muehlenhoff.

@Muehlenhoff ^

Dzahn added a subtask: T224583: Migrate labstore1006/1007 to Stretch/Buster.Mar 14 2020, 12:12 AM
Dzahn added subtasks: T216815: Upgrade Thumbor to Buster, T231067: Install Debian Buster on Hadoop, T213089: Upgrade memcached cluster to Debian Buster, T224576: Upgrade install servers to Buster, T236606: Rebuild Toolforge elasticsearch cluster with Stretch or Buster, T245272: Draft a plan for upgrading kubernetes machines to buster, Restricted Task, T245757: Upgrade MediaWiki clusters to Debian Buster (debian 10), T246604: Install 1 buster+10.4 host per section.Mar 14 2020, 12:19 AM
Bstorm closed subtask T224583: Migrate labstore1006/1007 to Stretch/Buster as Resolved.Mar 14 2020, 12:26 AM
Dzahn mentioned this in T224549: Track remaining jessie systems in production.Mar 14 2020, 12:28 AM
Dzahn added a comment.Mar 17 2020, 6:12 PM

@Jdforrester-WMF I noticed the "metal" in the ticket title after adding VMs. But i guess i would argue that metal or VM does not matter for the need to upgrade and security.

Jdforrester-WMF added a comment.Mar 17 2020, 6:38 PM
In T247045#5976882, @Dzahn wrote:

@Jdforrester-WMF I noticed the "metal" in the ticket title after adding VMs. But i guess i would argue that metal or VM does not matter for the need to upgrade and security.

Yeah, it was mostly from the "SRE need to be involved" side that bare-metal requires but sometimes VMs can be converted in user land (e.g. the CI VM instances). Happy for the task to be used as is most useful for SRE. :-)

Marostegui closed subtask T246604: Install 1 buster+10.4 host per section as Resolved.Mar 18 2020, 9:40 AM
• JHedden removed a subtask: T236606: Rebuild Toolforge elasticsearch cluster with Stretch or Buster.Apr 1 2020, 1:30 PM
Dzahn closed subtask T247651: upgrade planet.wikimedia.org backends to buster as Resolved.Apr 2 2020, 11:41 AM
Dzahn added a subtask: T224591: Migrate contint* hosts to Buster.Apr 2 2020, 7:45 PM
Dzahn closed subtask T247648: miscweb1001/2001 - upgrade to buster or decom as Resolved.Apr 7 2020, 9:02 AM
jcrespo added a subtask: T224589: Migrate dbmonitor hosts to Buster.Apr 7 2020, 9:04 AM
Dzahn renamed this task from Migrate all of production metal to Buster or later to Migrate all of production metal and VMs to Buster or later.Apr 7 2020, 9:06 AM
jcrespo added a subtask: T236406: Switchover backup director service from helium to backup1001.Apr 7 2020, 9:07 AM
jcrespo added a subtask: T250666: Upgrade WMF database-and-backup-related hosts to buster.Apr 20 2020, 9:51 AM
Dzahn closed subtask T224576: Upgrade install servers to Buster as Resolved.Apr 27 2020, 2:15 PM
Dzahn closed subtask T247650: replace bromine and vega with buster VMs as Resolved.May 1 2020, 11:55 AM
Dzahn closed subtask T247649: upgrade people.wikimedia.org backend to buster as Resolved.May 21 2020, 10:42 AM
Volans reopened subtask T247649: upgrade people.wikimedia.org backend to buster as Open.May 22 2020, 9:49 AM
Volans closed subtask T247649: upgrade people.wikimedia.org backend to buster as Resolved.May 22 2020, 10:09 AM
MoritzMuehlenhoff closed subtask T245114: Migrate Cumin hosts to Buster as Resolved.Jun 22 2020, 1:10 PM
Dzahn closed subtask T224591: Migrate contint* hosts to Buster as Resolved.Jun 25 2020, 7:32 PM
Tgr mentioned this in T256822: [EPIC] Upgrade MediaWiki-Vagrant to Debian Buster.Jun 30 2020, 10:11 PM
MSantos closed subtask Restricted Task as Resolved.Jul 13 2020, 1:08 PM
fgiunchedi closed subtask T247967: Migrate role::netmon to Buster as Resolved.Jul 30 2020, 9:06 AM
fgiunchedi closed subtask T247968: Migrate logging::webrequest::ops to Buster as Resolved.Jul 31 2020, 9:07 AM
Dzahn closed subtask T247652: replace backends for releases.wikimedia.org with buster VMs as Resolved.Aug 18 2020, 10:24 PM
JMeybohm closed subtask T239835: setup new, buster based, kubernetes etcd servers for staging/codfw/eqiad cluster as Resolved.Sep 16 2020, 8:08 AM
Gehel added a subtask: T264292: Migrate maps to Buster.Oct 1 2020, 10:08 AM
Krinkle added a subtask: T265643: Upgrade MediaWiki's Redis cluster to Debian Buster.Oct 15 2020, 7:23 PM
jijiki changed the status of subtask T213089: Upgrade memcached cluster to Debian Buster from Stalled to Open.Oct 20 2020, 8:44 PM
jijiki changed the status of subtask T245757: Upgrade MediaWiki clusters to Debian Buster (debian 10) from Stalled to Open.Oct 28 2020, 3:39 PM
Gehel closed subtask T244753: Migrate WDQS to Debian Buster as Resolved.Jan 6 2021, 9:01 AM
jijiki closed subtask T213089: Upgrade memcached cluster to Debian Buster as Resolved.Jan 13 2021, 3:34 PM
jijiki closed subtask T265643: Upgrade MediaWiki's Redis cluster to Debian Buster as Resolved.Jan 13 2021, 3:37 PM
Dzahn added a subtask: T272550: Upgrade docker-registry servers to Debian Buster.Jan 20 2021, 10:28 PM
Legoktm changed the status of subtask T245757: Upgrade MediaWiki clusters to Debian Buster (debian 10) from Open to Stalled.Jan 30 2021, 9:11 AM
Legoktm changed the status of subtask T245757: Upgrade MediaWiki clusters to Debian Buster (debian 10) from Stalled to Open.Feb 4 2021, 12:10 AM
akosiaris closed subtask T249724: Track and remove jessie based container images from production as Resolved.Feb 9 2021, 10:24 AM
elukey changed the status of subtask T231067: Install Debian Buster on Hadoop from Stalled to Open.Feb 18 2021, 7:29 AM
Legoktm closed subtask T272550: Upgrade docker-registry servers to Debian Buster as Resolved.Mar 10 2021, 11:49 PM
hnowlan closed subtask T274119: Copy cassandra packages to buster-wikimedia as Resolved.Mar 12 2021, 12:39 PM
Marostegui changed the status of subtask T224589: Migrate dbmonitor hosts to Buster from Stalled to Open.Apr 5 2021, 8:29 AM
Dzahn changed the status of subtask T247653: replace doc1001.eqiad.wmnet with a buster VM and create the codfw equivalent from Open to Stalled.Apr 6 2021, 6:45 PM
MoritzMuehlenhoff closed subtask T224589: Migrate dbmonitor hosts to Buster as Resolved.Apr 14 2021, 11:43 AM
lmata closed subtask T247966: Migrate role::alerting_host to Buster as Resolved.Jun 14 2021, 3:47 PM
JoKalliauer added a subscriber: JoKalliauer.Jul 2 2021, 1:36 PM
• razzi closed subtask T231067: Install Debian Buster on Hadoop as Resolved.Jul 28 2021, 6:24 PM
Dzahn changed the status of subtask T245757: Upgrade MediaWiki clusters to Debian Buster (debian 10) from Open to Stalled.Aug 5 2021, 1:37 PM
Gehel closed subtask T244736: Migrate Elasticsearch to Debian Buster as Declined.Aug 23 2021, 3:11 PM
Jdforrester-WMF added a subtask: T289135: Upgrade Cirrus Elasticsearch clusters to Debian Bullseye.Aug 23 2021, 6:05 PM
Bstorm mentioned this in T289883: cloud-vps: actively deprecate/phase out use of Debian Stretch.Aug 30 2021, 9:18 PM
hnowlan closed subtask T264292: Migrate maps to Buster as Resolved.Sep 9 2021, 9:51 AM
Dzahn changed the status of subtask T245757: Upgrade MediaWiki clusters to Debian Buster (debian 10) from Stalled to In Progress.Sep 16 2021, 2:38 PM
Dzahn closed subtask T245757: Upgrade MediaWiki clusters to Debian Buster (debian 10) as Resolved.Sep 16 2021, 3:22 PM
fgiunchedi reopened subtask T245757: Upgrade MediaWiki clusters to Debian Buster (debian 10) as Open.Sep 24 2021, 8:33 AM
fgiunchedi closed subtask T247963: Migrate role::graphite::production to Bullseye as Resolved.Oct 26 2021, 11:45 AM
Legoktm closed subtask T245757: Upgrade MediaWiki clusters to Debian Buster (debian 10) as Resolved.Nov 4 2021, 7:10 PM
• taavi changed the status of subtask T247653: replace doc1001.eqiad.wmnet with a buster VM and create the codfw equivalent from Stalled to Open.Nov 24 2021, 9:00 PM
Dzahn changed the status of subtask T247653: replace doc1001.eqiad.wmnet with a buster VM and create the codfw equivalent from Open to Stalled.Nov 24 2021, 10:06 PM
Krinkle changed the status of subtask T247653: replace doc1001.eqiad.wmnet with a buster VM and create the codfw equivalent from Stalled to Open.Dec 7 2021, 7:11 PM
Marostegui closed subtask T250666: Upgrade WMF database-and-backup-related hosts to buster as Resolved.Dec 14 2021, 8:17 AM
fgiunchedi closed subtask T247962: Migrate role::prometheus to Bullseye as Resolved.Feb 16 2022, 8:38 AM
MoritzMuehlenhoff mentioned this in T302086: Set scap minimum python version to 3.7.Feb 18 2022, 5:01 PM
dancy added a subtask: T302086: Set scap minimum python version to 3.7.Feb 18 2022, 5:02 PM
• taavi removed a subtask: T302086: Set scap minimum python version to 3.7.Feb 18 2022, 7:04 PM
• taavi added a parent task: T302086: Set scap minimum python version to 3.7.
Jdforrester-WMF updated the task description. (Show Details)Feb 23 2022, 4:53 PM
Jdforrester-WMF mentioned this in T291916: Tracking task for Bullseye migrations in production.
Jdforrester-WMF updated the task description. (Show Details)Feb 24 2022, 2:22 PM
JMeybohm closed subtask T245272: Draft a plan for upgrading kubernetes machines to buster as Resolved.Apr 4 2022, 7:30 AM
Dzahn closed subtask T253779: Migrate role::bastionhost::general and role::bastionhost::pop to Buster as Resolved.Apr 26 2022, 7:59 PM
Dzahn removed a subtask: T289135: Upgrade Cirrus Elasticsearch clusters to Debian Bullseye.
Dzahn added a subtask: T289135: Upgrade Cirrus Elasticsearch clusters to Debian Bullseye.Apr 26 2022, 8:02 PM
Dzahn changed the status of subtask T247653: replace doc1001.eqiad.wmnet with a buster VM and create the codfw equivalent from Open to In Progress.Jun 30 2022, 10:11 PM
Dzahn closed subtask T247653: replace doc1001.eqiad.wmnet with a buster VM and create the codfw equivalent as Resolved.EditedJul 7 2022, 8:16 PM

doc1001 has just been deleted. fixed one of the few remaining subtask. might be a good time to check the others and see where we are.

edit: this is the current status:

127 hosts

aqs[1004-1009].eqiad.wmnet,cloudelastic[1001,1004-1005].wikimedia.org,conf[1004-1006].eqiad.wmnet,elastic[2025-2034,2037-2052,2054-2060].codfw.wmnet,elastic[1048-1083].eqiad.wmnet,labstore[1004-1005].eqiad.wmnet,ms-be[2028-2039].codfw.wmnet,ms-be[1028-1033,1035-1039].eqiad.wmnet,ms-fe2009.codfw.wmnet,ms-fe1009.eqiad.wmnet,rdb[1009-1010].eqiad.wmnet,restbase-dev[1004-1006].eqiad.wmnet,sessionstore[2001-2003].codfw.wmnet,sessionstore[1001-1003].eqiad.wmnet,thumbor[2003-2006].codfw.wmnet,thumbor[1001-1002,1005-1006].eqiad.wmnet

Dzahn mentioned this in T312595: ms-be2028 on stretch.Jul 7 2022, 9:18 PM
Dzahn added a subtask: T312595: ms-be2028 on stretch.
Dzahn closed subtask T312595: ms-be2028 on stretch as Invalid.Jul 7 2022, 9:52 PM
bking closed subtask T289135: Upgrade Cirrus Elasticsearch clusters to Debian Bullseye as Resolved.Aug 22 2022, 3:20 PM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL