Page MenuHomePhabricator
Create Task
Maniphest T240965

Enable private network interface on Ceph OSD and MON hosts
Closed, ResolvedPublic
Actions

Description

Currently the Ceph servers have their public interface active and we'd like to get the private interface online.

For reference, there was a lot of talk on this subject in the rack/setup/install task T224188.

The Ceph servers are all in EQIAD row B, racks b2, b4, and b7. https://netbox.wikimedia.org/search/?q=cloudceph

Each Ceph server has 2x10GB interfaces:

NICvlansubnetpurposenotes
NIC0public1-b-eqiad208.80.154.128/26Client read/write IOAll clients are located in row b, cloud-hosts1-b-eqiad
NIC1private1-b-eqiad10.64.16.0/22Inter-cluster replication and heartbeatAll communication will be on the local subnet

Details

SubjectRepoBranchLines +/-
ceph: add support for dedicated cluster networkoperations/puppetproduction+134 -101
ceph: add secondary interface for cloudceph serversoperations/dnsmaster+12 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

• JHedden created this task.Dec 17 2019, 4:35 PM
• JHedden added a subscriber: ayounsi.Dec 17 2019, 4:37 PM

@ayounsi and @Bstorm could you please review the vlan and subnet for the private interface? I think it's the right one but would like confirmation.

• JHedden claimed this task.Dec 17 2019, 4:38 PM
• JHedden moved this task from Inbox to Doing on the cloud-services-team (Kanban) board.
gerritbot added a comment.Dec 17 2019, 6:38 PM

Change 558636 had a related patch set uploaded (by Jhedden; owner: Jhedden):
[operations/dns@master] ceph: add secondary interface for cloudceph servers

https://gerrit.wikimedia.org/r/558636

gerritbot added a project: Patch-For-Review.Dec 17 2019, 6:38 PM
• Bstorm added a comment.Dec 17 2019, 6:44 PM

I believe that was what we discussed on the earlier tasks about racking: T224188#5243578
It's always reasonable to revisit if needed, but those are the specific vlans mentioned.

ayounsi added a comment.Dec 18 2019, 5:21 AM

Private is good but public is incorrect:
public1-b-eqiad:

ipv4: 208.80.154.128/26

private1-b-eqiad:

ipv4: 10.64.16.0/22

Those are defined in many places but that doc might be the easiest to use: https://github.com/wikimedia/puppet/blob/production/modules/network/data/data.yaml

• JHedden updated the task description. (Show Details)Dec 18 2019, 6:26 AM

Thanks for the review, I had the wrong subnet here but configured the hosts on the correct public 208.80.154.128/26 subnet.

gerritbot added a comment.Dec 19 2019, 11:12 PM

Change 559620 had a related patch set uploaded (by Jhedden; owner: Jhedden):
[operations/puppet@production] ceph: add support for dedicated cluster network

https://gerrit.wikimedia.org/r/559620

gerritbot added a comment.Dec 20 2019, 9:52 PM

Change 558636 merged by Jhedden:
[operations/dns@master] ceph: add secondary interface for cloudceph servers

https://gerrit.wikimedia.org/r/558636

gerritbot added a comment.Dec 23 2019, 3:47 PM

Change 559620 merged by Jhedden:
[operations/puppet@production] ceph: add support for dedicated cluster network

https://gerrit.wikimedia.org/r/559620

• JHedden closed this task as Resolved.Dec 23 2019, 4:03 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL