Page MenuHomePhabricator

Enable private network interface on Ceph OSD and MON hosts
Closed, ResolvedPublic

Description

Currently the Ceph servers have their public interface active and we'd like to get the private interface online.

For reference, there was a lot of talk on this subject in the rack/setup/install task T224188.

The Ceph servers are all in EQIAD row B, racks b2, b4, and b7. https://netbox.wikimedia.org/search/?q=cloudceph

Each Ceph server has 2x10GB interfaces:

NICvlansubnetpurposenotes
NIC0public1-b-eqiad208.80.154.128/26Client read/write IOAll clients are located in row b, cloud-hosts1-b-eqiad
NIC1private1-b-eqiad10.64.16.0/22Inter-cluster replication and heartbeatAll communication will be on the local subnet

Details

Related Gerrit Patches:

Event Timeline

JHedden created this task.Dec 17 2019, 4:35 PM

@ayounsi and @Bstorm could you please review the vlan and subnet for the private interface? I think it's the right one but would like confirmation.

JHedden claimed this task.Dec 17 2019, 4:38 PM
JHedden moved this task from Inbox to Doing on the cloud-services-team (Kanban) board.

Change 558636 had a related patch set uploaded (by Jhedden; owner: Jhedden):
[operations/dns@master] ceph: add secondary interface for cloudceph servers

https://gerrit.wikimedia.org/r/558636

I believe that was what we discussed on the earlier tasks about racking: T224188#5243578
It's always reasonable to revisit if needed, but those are the specific vlans mentioned.

Private is good but public is incorrect:
public1-b-eqiad:

ipv4: 208.80.154.128/26

private1-b-eqiad:

ipv4: 10.64.16.0/22

Those are defined in many places but that doc might be the easiest to use: https://github.com/wikimedia/puppet/blob/production/modules/network/data/data.yaml

JHedden updated the task description. (Show Details)Dec 18 2019, 6:26 AM

Thanks for the review, I had the wrong subnet here but configured the hosts on the correct public 208.80.154.128/26 subnet.

Change 559620 had a related patch set uploaded (by Jhedden; owner: Jhedden):
[operations/puppet@production] ceph: add support for dedicated cluster network

https://gerrit.wikimedia.org/r/559620

Change 558636 merged by Jhedden:
[operations/dns@master] ceph: add secondary interface for cloudceph servers

https://gerrit.wikimedia.org/r/558636

Change 559620 merged by Jhedden:
[operations/puppet@production] ceph: add support for dedicated cluster network

https://gerrit.wikimedia.org/r/559620

JHedden closed this task as Resolved.Dec 23 2019, 4:03 PM