labstore1004/5 - Reconfigure the DRBD interfaces to run across the 10G cable
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Bstorm
	May 12 2021, 10:29 PM

Description

The second 10G interfaces on the primary cluster are now confirmed connected. We need to figure out changing the interface the DBRD devices use as safely as possible with minimal downtime so that write traffic can operate at full speed on the cluster.

For test, 192.168.0.3 and 4 are assigned to the interfaces (both enp133s0f1). Those likely need to switch to the existing numbers without corrupting the data or something.

Details

Subject	Repo	Branch	Lines +/-
cloud nfs: fix the netmask for swapping cables	operations/puppet	production	+1 -1
labstore: Switch DRBD devices to using the 10Gb addresses	operations/puppet	production	+4 -4
labstore: Switch DRBD devices to using the 10Gb addresses	operations/puppet	production	+4 -4

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	• Bstorm	T266198 Move labstore1004 and labstore1005 to 10G Ethernet
Resolved	• Bstorm	T282754 labstore1004/5 - Reconfigure the DRBD interfaces to run across the 10G cable
Invalid	Jclark-ctr	T282799 labstore1004/5: buy a DAC 10Gb cable or adjust the current fiber cable for DAC/crossover

Event Timeline

• Bstorm created this task.May 12 2021, 10:29 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 12 2021, 10:29 PM

• Bstorm moved this task from Backlog to Shared Storage on the Data-Services board.May 12 2021, 10:30 PM

Since I've already added new IPs...this might be extremely easy: https://stackoverflow.com/questions/61909137/whats-the-step-to-change-drbd-ip

Change 690563 had a related patch set uploaded (by Bstorm; author: Bstorm):

[operations/puppet@production] labstore: Switch DRBD devices to using the 10Gb addresses

https://gerrit.wikimedia.org/r/690563

gerritbot added a project: Patch-For-Review.May 13 2021, 4:26 PM

Change 690563 merged by Bstorm:

[operations/puppet@production] labstore: Switch DRBD devices to using the 10Gb addresses

https://gerrit.wikimedia.org/r/690563

Maintenance_bot removed a project: Patch-For-Review.May 13 2021, 5:10 PM

Drat. It's not going to be this easy. I believe there still needs to be a "crossover" adjustment to the cable end to make this work. Reverting and making a subtask.