Page MenuHomePhabricator

Q1:rack/setup/install fransw200[1-3].frack.codfw.wmnet
Closed, ResolvedPublic

Description

This task will track the racking, setup, and OS installation of fransw200[1-3].frack.codfw.wmnet

Hostname / Racking / Installation Details

Hostnames: fransw200[1-3].frack.codfw.wmnet
Racking Proposal: frack codfw
Networking Setup: # of Connections:2 - Speed:10G. - VLAN:frack-fundraising : AAAA records:Y/N, Additional IP records (Cassandra)? Yes/No
Partitioning/Raid: n/a
OS Distro: n/a
Sub-team Technical Contact: Dallas Wisehaupt or Jeff Green

Per host setup checklist

fransw2001:
  • Receive in system on procurement task T367609 & in Coupa
  • Rack system with proposed racking plan (see above) & update Netbox (include all system info plus location, state of planned)
  • Run the Provision a server's network attributes Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
  • Immediately run the sre.dns.netbox cookbook
  • Immediately run the sre.hosts.provision cookbook
  • Run the sre.hardware.upgrade-firmware cookbook
  • Update the operations/puppet repo - this should include updates to preseed.yaml, and site.pp with roles defined by service group: https://wikitech.wikimedia.org/wiki/SRE/Dc-operations
  • Run the sre.hosts.reimage cookbook
fransw2002:
  • Receive in system on procurement task T367609 & in Coupa
  • Rack system with proposed racking plan (see above) & update Netbox (include all system info plus location, state of planned)
  • Run the Provision a server's network attributes Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
  • Immediately run the sre.dns.netbox cookbook
  • Immediately run the sre.hosts.provision cookbook
  • Run the sre.hardware.upgrade-firmware cookbook
  • Update the operations/puppet repo - this should include updates to preseed.yaml, and site.pp with roles defined by service group: https://wikitech.wikimedia.org/wiki/SRE/Dc-operations
  • Run the sre.hosts.reimage cookbook
fransw2003:
  • Receive in system on procurement task T367609 & in Coupa
  • Rack system with proposed racking plan (see above) & update Netbox (include all system info plus location, state of planned)
  • Run the Provision a server's network attributes Netbox script - Note that you must run the DNS and Provision cookbook after completing this step
  • Immediately run the sre.dns.netbox cookbook
  • Immediately run the sre.hosts.provision cookbook
  • Run the sre.hardware.upgrade-firmware cookbook
  • Update the operations/puppet repo - this should include updates to preseed.yaml, and site.pp with roles defined by service group: https://wikitech.wikimedia.org/wiki/SRE/Dc-operations
  • Run the sre.hosts.reimage cookbook

Event Timeline

RobH mentioned this in Unknown Object (Task).Jun 17 2024, 6:12 PM
RobH added a parent task: Unknown Object (Task).
RobH moved this task from Backlog to Racking Tasks on the ops-codfw board.
Jgreen added a parent task: Restricted Task.Jun 26 2024, 1:12 AM
Jhancock.wm added subscribers: Papaul, Jhancock.wm.

@Papaul these are ready for your part.

fransw2001
ETH0 <-> FASW-C8A eth-0/0/29
ETH1 <-> FASW-C8B eth-1/0/29

fransw2002
ETH0 <-> FASW-C8A eth-0/0/30
ETH1 <-> FASW-C8B eth-1/0/30

fransw2003
ETH0 <-> FASW-C8A eth-0/0/31
ETH1 <-> FASW-C8B eth-1/0/31

[edit interfaces interface-range disabled]
-    member ge-0/0/29;
-    member ge-0/0/30;
-    member ge-0/0/31;
-    member ge-1/0/29;
-    member ge-1/0/30;
-    member ge-1/0/31;
[edit interfaces interface-range vlan-fundraising]
     member "ge-[0-1]/0/32" { ... }
+    member "ge-[0-1]/0/29";
+    member "ge-[0-1]/0/30";
+    member "ge-[0-1]/0/31";
[edit interfaces]
+   ge-0/0/29 {
+       description fransw2001:eth0;
+   }
+   ge-0/0/30 {
+       description fransw2002:eth0;
+   }
+   ge-0/0/31 {
+       description fransw2003:eth0;
+   }
+   ge-1/0/29 {
+       description fransw2001:eth1;
+   }
+   ge-1/0/30 {
+       description fransw2002:eth1;
+   }
+   ge-1/0/31 {
+       description fransw2003:eth1;
+   }
{master:1}[edit]
papaul@fasw-c-codfw# run show interfaces ge-[0-1]/0/29 descriptions 
Interface       Admin Link Description
ge-0/0/29       up    down fransw2001:eth0
ge-1/0/29       up    up   fransw2001:eth1

{master:1}[edit]
papaul@fasw-c-codfw# run show interfaces ge-[0-1]/0/30 descriptions    
Interface       Admin Link Description
ge-0/0/30       up    up   fransw2002:eth0
ge-1/0/30       up    up   fransw2002:eth1

{master:1}[edit]
papaul@fasw-c-codfw# run show interfaces ge-[0-1]/0/31 descriptions    
Interface       Admin Link Description
ge-0/0/31       up    up   fransw2003:eth0
ge-1/0/31       up    up   fransw2003:eth1

@Jhancock.wm when you back onsite can you please check fransw2001 first interface it is showing link down. Thanks

Change #1057023 had a related patch set uploaded (by Papaul; author: Papaul):

[operations/dns@master] Add new frack nodes to DNS files

https://gerrit.wikimedia.org/r/1057023

Change #1057023 merged by Papaul:

[operations/dns@master] Add new frack nodes to DNS files

https://gerrit.wikimedia.org/r/1057023

@Papaul @Jhancock.wm I'm getting to building these hosts now (so many other things were pre-reqs) and they are starting out ok. Except fransw2002 is not reachable on the drac. Additionally, it's still listed in rack C7 instead of rack C8.

Could we have someone check on fransw2002? Feel free to do anything that's needed to it as it's not in any service yet.

@Dwisehaupt yes the host is still racked in C7 we are waiting for civic2001 to be decom so we can move it into C8/U16. For the issue about the host is not reachable i think we forgot to plug the mgmt cable we will check that tomorrow when we are back on site.
@Jhancock.wm if you get on site before me tomorrow can you please plug the mgmt cable for this server? Thank you

@Papaul Thanks. I'll take a look at civi2001. I believe we need to keep it in place through the end of the month (just in case for big english) and then we can decommission it.

@Dwisehaupt there is no rush on our end. You can take you time on that,

@Dwisehaupt @Papaul got the cable reconnected and confirmed it pings. if there are any issues with it, lmk and I'll take care of it asap.

Dwisehaupt moved this task from Backlog to Done on the fundraising-tech-ops board.

Hosts are built out and in the configuration stages which is covered in other tasks. Closing.

Change #1140541 had a related patch set uploaded (by Dwisehaupt; author: Dwisehaupt):

[operations/puppet@production] icinga: add new frack hosts for basic monitoring

https://gerrit.wikimedia.org/r/1140541

Change #1140541 merged by Filippo Giunchedi:

[operations/puppet@production] icinga: add new frack hosts for basic monitoring

https://gerrit.wikimedia.org/r/1140541