Page MenuHomePhabricator
Create Task
Maniphest T286879

lvs2007, lvs2009, lvs2010 should not be on the same row A switch
Closed, ResolvedPublic
Actions

Description

During the crash of row A2 switch on codfw T286787 it's been detected that lvs2007, lvs2009 and lvs2010 get their row A traffic from the very same switch (A2):

xe-2/0/43                  lvs2009
xe-2/0/44                  lvs2010
xe-2/0/45                  lvs2007

this is pretty bad for lvs2007 and lvs2010 and we should move lvs2010 row A NIC to another switch ASAP

Follows-up https://wikitech.wikimedia.org/wiki/Incident_documentation/2021-07-16_codfw_network

Related Objects
Search...

Event Timeline

Vgutierrez created this task.Jul 19 2021, 8:24 AM
Restricted Application added a project: Infrastructure-Foundations. · View Herald TranscriptJul 19 2021, 8:24 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Vgutierrez moved this task from Backlog to Network on the Traffic board.Jul 19 2021, 8:25 AM
Vgutierrez edited projects, added DC-Ops, ops-codfw; removed netops, Infrastructure-Foundations.
ayounsi subscribed.Jul 19 2021, 8:28 AM

Other rows need to be audited as well.

taavi added a project: Sustainability (Incident Followup).Jul 19 2021, 8:29 AM
Vgutierrez added a comment.Jul 19 2021, 8:43 AM
In T286879#7220243, @ayounsi wrote:

Other rows need to be audited as well.

You're right, I've created T286881 to do so.

Maintenance_bot added a project: SRE.Jul 19 2021, 8:45 AM
Vgutierrez added a parent task: T286881: Audit eqiad & codfw LVS network links.Jul 19 2021, 8:47 AM
Legoktm triaged this task as High priority.Jul 26 2021, 11:15 PM
Papaul closed this task as Resolved.Aug 2 2021, 3:38 PM
Papaul claimed this task.
Papaul subscribed.

Resolving this task since all 3 links on lvs2010 moved to asw-a4,asw-b4 and asw-c4

Krinkle renamed this task from lvs2007, lvs2009 and lvs2010 connected to the same row A switch to lvs2007, lvs2009, lvs2010 should not be on the same row A switch.Aug 18 2021, 4:47 AM
Krinkle updated the task description. (Show Details)
BBlack moved this task from Network to Done on the Traffic board.Oct 8 2021, 5:59 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits