Page MenuHomePhabricator

(No Need By Date Provided) rack/setup/install restbase202[123]
Closed, ResolvedPublic

Description

This task tracks the setup of 3 new restbase nodes in codfw:restbase202[123]

Hostnames: restbase202[123]
Racking Proposal: Place 1 each in 1G rack in B, C, and D. per T238580#5710739
Networking/Subnet/VLAN/IP: single 1G production network connection, match other restbase nodes.
Partitioning/Raid: match existing restbase nodes

restbase2021: Row B rack B1 ge-1/0/16

  • - receive in system on procurement task T238579
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

restbase2022: Row C rack C1 ge-1/0/12

  • - receive in system on procurement task T238579
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

restbase2023: Row D rack D8 ge-8/0/6

  • - receive in system on procurement task T238579
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

Details

Related Gerrit Patches:

Event Timeline

RobH triaged this task as Medium priority.Jan 2 2020, 9:33 PM
RobH created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 2 2020, 9:33 PM
Papaul moved this task from Backlog to Racking Tasks on the ops-codfw board.Jan 3 2020, 1:44 AM
wiki_willy renamed this task from rack/setup/install restbase202[123] to (No Need By Date Provided) rack/setup/install restbase202[123].Jan 3 2020, 6:54 PM
Eevans updated the task description. (Show Details)Jan 6 2020, 4:57 PM
In T238580#5710739, @Eevans wrote:
In T238580#5709953, @RobH wrote:

Also note I assumed details for the racking/hostnames and would appreciate confirmation of those details in task description, thanks!

This cluster uses a replication count of 3 (per-DC), and for eqiad we have machines evenly distributed over a, b, and d. This replica-to-row affinity makes it very nice to reason about where data will be moving from/to on topology changes and it would be a shame if we lost that now. Will there be a problem keeping these to the same 3 rows currently in-use?

So the racking plan needs to adjust, one in B, C, and D.

Papaul updated the task description. (Show Details)Jan 6 2020, 6:49 PM
RobH updated the task description. (Show Details)Jan 6 2020, 7:39 PM
Papaul updated the task description. (Show Details)Jan 7 2020, 8:11 PM
Papaul updated the task description. (Show Details)Jan 14 2020, 4:57 PM
Papaul updated the task description. (Show Details)Jan 14 2020, 5:50 PM

Change 565048 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/dns@master] DNS: Add mgmt and producion DNS for restbase202[1-3]

https://gerrit.wikimedia.org/r/565048

Change 565057 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/puppet@production] DHCP: Add MAC address entires for restbase202[1-3]

https://gerrit.wikimedia.org/r/565057

Change 565048 merged by Filippo Giunchedi:
[operations/dns@master] DNS: Add mgmt and producion DNS for restbase202[1-3]

https://gerrit.wikimedia.org/r/565048

Papaul updated the task description. (Show Details)Jan 15 2020, 4:32 PM

Change 565057 merged by Filippo Giunchedi:
[operations/puppet@production] DHCP: Add MAC address entires for restbase202[1-3]

https://gerrit.wikimedia.org/r/565057

Change 565058 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/puppet@production] Partman: Add restbase202[1-3] to netboot.cfg

https://gerrit.wikimedia.org/r/565058

Change 565058 merged by Filippo Giunchedi:
[operations/puppet@production] Partman: Add restbase202[1-3] to netboot.cfg

https://gerrit.wikimedia.org/r/565058

papaul@asw-b-codfw# show | compare 
[edit interfaces interface-range vlan-private1-b-codfw]
     member ge-5/0/1 { ... }
+    member ge-1/0/16;
[edit interfaces interface-range disabled]
-    member ge-1/0/16;
[edit interfaces]
+   ge-1/0/16 {
+       description restbase2021;
+   }
papaul@asw-b-codfw# run show interfaces ge-1/0/16 descriptions 
Interface       Admin Link Description
ge-1/0/16       up    up   restbase2021
papaul@asw-c-codfw# show | compare 
[edit interfaces interface-range vlan-private1-c-codfw]
     member xe-7/0/11 { ... }
+    member ge-1/0/12;
[edit interfaces interface-range disabled]
-    member ge-1/0/12;
[edit interfaces]
+   ge-1/0/12 {
+       description restbase2022;
+   }
apaul@asw-c-codfw> show interfaces ge-1/0/12 descriptions 
Interface       Admin Link Description
ge-1/0/12       up    up   restbase2022
apaul@asw-d-codfw# show | compare 
[edit interfaces interface-range vlan-private1-d-codfw]
     member xe-4/0/15 { ... }
+    member ge-8/0/6;
[edit interfaces interface-range disabled]
-    member ge-8/0/6;
[edit interfaces]
+   ge-8/0/6 {
+       description restbase2023;
+   }
papaul@asw-d-codfw# run show interfaces ge-8/0/6 descriptions 
Interface       Admin Link Description
ge-8/0/6        up    up   restbase2023
Papaul updated the task description. (Show Details)Jan 15 2020, 4:54 PM
Papaul updated the task description. (Show Details)Jan 15 2020, 5:24 PM
Papaul updated the task description. (Show Details)Jan 15 2020, 8:34 PM
Papaul reassigned this task from Papaul to fgiunchedi.Jan 15 2020, 9:32 PM
Papaul updated the task description. (Show Details)
Papaul added a subscriber: Papaul.

@fgiunchedi all yours

Change 565281 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] Add restbase2[123] to restbase::production

https://gerrit.wikimedia.org/r/565281

Change 565281 merged by Filippo Giunchedi:
[operations/puppet@production] Add restbase2[123] to restbase::production

https://gerrit.wikimedia.org/r/565281

Change 565305 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/dns@master] Add cassandra instances for restbase202[123]

https://gerrit.wikimedia.org/r/565305

Change 565305 merged by Filippo Giunchedi:
[operations/dns@master] Add cassandra instances for restbase202[123]

https://gerrit.wikimedia.org/r/565305

Change 565307 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] hieradata: add restbase202[123] instances

https://gerrit.wikimedia.org/r/565307

Change 565307 merged by Filippo Giunchedi:
[operations/puppet@production] hieradata: add restbase202[123] instances

https://gerrit.wikimedia.org/r/565307

fgiunchedi closed this task as Resolved.Jan 17 2020, 9:15 AM
fgiunchedi reassigned this task from fgiunchedi to Eevans.
fgiunchedi updated the task description. (Show Details)

All done, service is being implemented in T243000