[cloudvirt] Move to jumbo frames
Open, HighPublic
Actions

Assigned To

Authored By

	dcaro
	Feb 20 2023, 12:51 PM

Description

We currently have our ceph osd hosts setup to use jumbo frames, but the cloudvirts are configured to use 1500 MTU.

This is not optimal as jumbo frames will allow for higher throughput to the VM storage layer (ceph).

All the switches (cloudsw*) are configured with jumbo frames too.

This task is to enable the jumbo frames on the cloudvirts making sure nothing breaks in-between.

Related Objects

Mentioned In: T329711: [ceph] Add monitoring for inter-osd/mon/cloudvirt connectivity
Mentioned Here: T319184: Move WMCS servers to 1 single NIC
T329711: [ceph] Add monitoring for inter-osd/mon/cloudvirt connectivity
T329778: [ceph] Investigate if there's a way to degrade instead of failing when jumbo frames are being dropped in the network

Event Timeline

dcaro triaged this task as High priority.Feb 20 2023, 12:51 PM

dcaro created this task.

Questions for NetOps: they live in the cloud-hosts vlan. It is OK if some hosts attached to that VLAN use high MTU and other don't?

dcaro mentioned this in T329711: [ceph] Add monitoring for inter-osd/mon/cloudvirt connectivity.Feb 20 2023, 2:01 PM

aborrero added a project: netops.Feb 21 2023, 5:14 PM

aborrero added subscribers: ayounsi, cmooney.

Restricted Application added a project: Infrastructure-Foundations. · View Herald TranscriptFeb 21 2023, 5:14 PM

In T330075#8629532, @aborrero wrote:

Questions for NetOps: they live in the cloud-hosts vlan. It is OK if some hosts attached to that VLAN use high MTU and other don't?

TCP handshake contains MSS data of each side so they will both send packets at the max size they can send and max size the other side can receive.
And they can be asymmetric, host A can send packets at 9000 to host B, and host B send 1500 packets to host A.

UDP by default is also < 1500.

So for UDP it comes down to the application, and TCP should do the right thing.

Ok, thanks!

Next question would be:

Because T319184: Move WMCS servers to 1 single NIC some cloudvirts have a single NIC trunked with 1 native + 1 tagged VLAN. I wonder if there is any problem if each have a different MTU set on either the linux side or the switch port side.

Switch side is already at 9192.

Server TCP stack will define the MSS based on the interface used to reach the other endpoint (so based on the routing table).

So here too it should just work.

fnegri subscribed.Feb 22 2023, 2:52 PM

ayounsi moved this task from Backlog to Watching on the netops board.Mar 16 2023, 9:11 PM

• nskaggs removed a project: cloud-services-team (FY2022/2023-Q3).Apr 28 2023, 1:53 PM

joanna_borun removed projects: Infrastructure-Foundations, netops.Feb 12 2024, 3:51 PM