Page MenuHomePhabricator

Move all cloud-vps VMs to Ceph
Closed, ResolvedPublic

Description

  • toolforge worker nodes rebuilt on or moved to ceph
  • new ceph-enabled flavors added to replace existing flavors
  • move VMs from cloudvirt1001-1009 to ceph, mark cloudvirt1001-1009 for decom
  • convert remaining cloudvirts to ceph
  • cheer

Related Objects

StatusSubtypeAssignedTask
OpenNone
OpenAndrew
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenAndrew
OpenAndrew
OpenNone
ResolvedAndrew
ResolvedAndrew
ResolvedCmjohnson
ResolvedAndrew
ResolvedAndrew
ResolvedAndrew
ResolvedAndrew
ResolvedAndrew
OpenAndrew
Resolvedaborrero
Opendcaro
ResolvedAndrew
ResolvedAndrew
ResolvedCmjohnson
ResolvedJclark-ctr
Resolvedayounsi
OpenNone

Event Timeline

Andrew created this task.Aug 24 2020, 3:05 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 24 2020, 3:05 PM
Andrew updated the task description. (Show Details)Sep 2 2020, 6:29 PM
Andrew triaged this task as High priority.Sep 8 2020, 4:15 PM
Andrew moved this task from Inbox to Soon! on the cloud-services-team (Kanban) board.
Andrew updated the task description. (Show Details)Sep 17 2020, 3:55 PM

A few special cases:

clouddb-wikilabels-01.clouddb-services.eqiad1.wikimedia.cloud and clouddb-wikilabels-02.clouddb-services.eqiad1.wikimedia.cloud should be moved to Ceph but they'll need special flavors -- much smaller disk than currently allocated and also probably a lifted IO throttle. If possible coordinate this with the wikilabels folks.

clouddb-100[1-4] can stay on local storage for now but their hypervisors need to be rebuild with Buster.

Change 634545 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[openstack/horizon/horizon@train] WMF Hack: hide non-ceph flavors from the instance creation workflow

https://gerrit.wikimedia.org/r/634545

Change 634545 merged by Andrew Bogott:
[openstack/horizon/horizon@train] WMF Hack: hide non-ceph flavors from the instance creation workflow

https://gerrit.wikimedia.org/r/634545

Change 635893 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] wmcs: change cloudvirt1025, 26 and 29 to ceph-enabled hypervisors

https://gerrit.wikimedia.org/r/635893

Change 635893 merged by Andrew Bogott:
[operations/puppet@production] wmcs: change cloudvirt1025, 26 and 29 to ceph-enabled hypervisors

https://gerrit.wikimedia.org/r/635893

Andrew updated the task description. (Show Details)Oct 28 2020, 2:13 AM
Andrew updated the task description. (Show Details)

Other than the special-case VMs in clouddb-services and wdqs VMs (which may remain with local storage for good), all VMs are now hosted on Ceph. Cloudvirt1025-30 are awaiting network upgrades before being rebuilt as Ceph-enabled hypervisors.

Andrew closed this task as Resolved.Wed, Nov 18, 7:51 PM

I'm closing this; upgrading and moving remaining hardware can be dealt with via different tasks.