Page MenuHomePhabricator

Ceph dev cluster in codfw1dev
Closed, ResolvedPublic

Description

I'm pleased to notice that we have three smallish servers (cloudceph200[1-3]-dev) racked in codfw1dev dedicated to Ceph development. Let's get them up and running and backing the -dev nova and glance services.

Either we can make each node a combined mon and osd node, or we can repurpose labtestpuppetmaster2001 (T258103) as a single mon node. Let's try the former unless applying the two puppet roles in combination causes trouble.

Right now those nodes are on public IPs. In order to replicate eqiad1 we need to create a cluster network for the backplane and then rebuild these nodes on the VM vlan (probably cloud-hosts1-b-codfw).

Event Timeline

Andrew added a subtask: Unknown Object (Task).Oct 20 2020, 2:16 AM

@aborrero and @ayounsi, how much work is involved in preparing vlans and configuring these hosts to talk to codfw1-dev VMs? Is it something we can do soon or should we set this aside for a future quarter?

I'd say next Q but DCops can maybe help here as well, at least to move the primary interface to the cloud-hosts vlan/re-image/etc.

nskaggs triaged this task as Medium priority.Oct 20 2020, 4:19 PM

Change 643316 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Roles and hiera for codfw1dev ceph cluster

https://gerrit.wikimedia.org/r/643316

Change 643316 merged by Andrew Bogott:
[operations/puppet@production] Roles and hiera for codfw1dev ceph cluster

https://gerrit.wikimedia.org/r/643316

Change 644257 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Ceph/codfw: added some missing hiera values

https://gerrit.wikimedia.org/r/644257

Change 644257 merged by Andrew Bogott:
[operations/puppet@production] Ceph/codfw: added some missing hiera values

https://gerrit.wikimedia.org/r/644257

Change 644584 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] cloudvirt2003-dev: move to ceph-enabled virt role

https://gerrit.wikimedia.org/r/644584

Change 644584 merged by Andrew Bogott:
[operations/puppet@production] cloudvirt2003-dev: move to ceph-enabled virt role, add ceph hiera elsewhere

https://gerrit.wikimedia.org/r/644584

Change 645213 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Make cloudvirt2002-dev ceph-enabled

https://gerrit.wikimedia.org/r/645213

Change 645213 merged by Andrew Bogott:
[operations/puppet@production] Make cloudvirt2002-dev ceph-enabled

https://gerrit.wikimedia.org/r/645213

I have a few more bumps to iron out but I have VMs hosted on the codfw1dev ceph cluster now.

Change 645223 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] nova compute: make live_migration_uri dc-specific

https://gerrit.wikimedia.org/r/645223

Change 645223 merged by Andrew Bogott:
[operations/puppet@production] nova compute: make live_migration_uri dc-specific

https://gerrit.wikimedia.org/r/645223

Change 645224 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Cloudvirt2001-dev make a ceph-backed hypervisor

https://gerrit.wikimedia.org/r/645224

Change 645224 merged by Andrew Bogott:
[operations/puppet@production] Cloudvirt2001-dev make a ceph-backed hypervisor

https://gerrit.wikimedia.org/r/645224

Change 645399 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Glance: enable rbd for glance in codfw1dev

https://gerrit.wikimedia.org/r/645399

Change 645399 merged by Andrew Bogott:
[operations/puppet@production] Glance: enable rbd for glance in codfw1dev

https://gerrit.wikimedia.org/r/645399

Change 647107 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Ceph: allow controller nodes to be specified per DC

https://gerrit.wikimedia.org/r/647107

Change 647107 merged by Andrew Bogott:
[operations/puppet@production] Ceph: allow controller nodes to be specified per DC

https://gerrit.wikimedia.org/r/647107

Change 647321 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[labs/private@master] Fake keydata for cinder ceph client

https://gerrit.wikimedia.org/r/647321

Change 647321 merged by Andrew Bogott:
[labs/private@master] Fake keydata for cinder ceph client

https://gerrit.wikimedia.org/r/647321

This is built and working.

Change 667920 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] nova compute: make live_migration_uri dc-specific

https://gerrit.wikimedia.org/r/667920

Change 667920 merged by Andrew Bogott:
[operations/puppet@production] nova compute: make live_migration_uri dc-specific

https://gerrit.wikimedia.org/r/667920