Page MenuHomePhabricator
Create Task
Maniphest T241790

(No Need By Date Provided) rack/setup/install restbase202[123]
Closed, ResolvedPublic
Actions

Description

This task tracks the setup of 3 new restbase nodes in codfw:restbase202[123]

Hostnames: restbase202[123]
Racking Proposal: Place 1 each in 1G rack in B, C, and D. per T238580#5710739
Networking/Subnet/VLAN/IP: single 1G production network connection, match other restbase nodes.
Partitioning/Raid: match existing restbase nodes

restbase2021: Row B rack B1 ge-1/0/16

  • - receive in system on procurement task T238579
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

restbase2022: Row C rack C1 ge-1/0/12

  • - receive in system on procurement task T238579
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

restbase2023: Row D rack D8 ge-8/0/6

  • - receive in system on procurement task T238579
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

Details

SubjectRepoBranchLines +/-
hieradata: add restbase202[123] instancesoperations/puppetproduction+108 -0
Add cassandra instances for restbase202[123]operations/dnsmaster+18 -0
Add restbase2[123] to restbase::productionoperations/puppetproduction+1 -1
Partman: Add restbase202[1-3] to netboot.cfgoperations/puppetproduction+1 -1
DHCP: Add MAC address entires for restbase202[1-3]operations/puppetproduction+15 -0
DNS: Add mgmt and producion DNS for restbase202[1-3]operations/dnsmaster+18 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

RobH triaged this task as Medium priority.Jan 2 2020, 9:33 PM
RobH created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 2 2020, 9:33 PM
Papaul moved this task from Backlog to Racking Tasks on the ops-codfw board.Jan 3 2020, 1:44 AM
wiki_willy renamed this task from rack/setup/install restbase202[123] to (No Need By Date Provided) rack/setup/install restbase202[123].Jan 3 2020, 6:54 PM
Eevans updated the task description. (Show Details)Jan 6 2020, 4:57 PM
In T238580#5710739, @Eevans wrote:
In T238580#5709953, @RobH wrote:

Also note I assumed details for the racking/hostnames and would appreciate confirmation of those details in task description, thanks!

This cluster uses a replication count of 3 (per-DC), and for eqiad we have machines evenly distributed over a, b, and d. This replica-to-row affinity makes it very nice to reason about where data will be moving from/to on topology changes and it would be a shame if we lost that now. Will there be a problem keeping these to the same 3 rows currently in-use?

So the racking plan needs to adjust, one in B, C, and D.

Papaul updated the task description. (Show Details)Jan 6 2020, 6:49 PM
RobH updated the task description. (Show Details)Jan 6 2020, 7:39 PM
Papaul updated the task description. (Show Details)Jan 7 2020, 8:11 PM
Papaul updated the task description. (Show Details)Jan 14 2020, 4:57 PM
Papaul updated the task description. (Show Details)Jan 14 2020, 5:50 PM
gerritbot added a comment.Jan 15 2020, 3:57 PM

Change 565048 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/dns@master] DNS: Add mgmt and producion DNS for restbase202[1-3]

https://gerrit.wikimedia.org/r/565048

gerritbot added a project: Patch-For-Review.Jan 15 2020, 3:57 PM
gerritbot added a comment.Jan 15 2020, 4:29 PM

Change 565057 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/puppet@production] DHCP: Add MAC address entires for restbase202[1-3]

https://gerrit.wikimedia.org/r/565057

gerritbot added a comment.Jan 15 2020, 4:30 PM

Change 565048 merged by Filippo Giunchedi:
[operations/dns@master] DNS: Add mgmt and producion DNS for restbase202[1-3]

https://gerrit.wikimedia.org/r/565048

Papaul updated the task description. (Show Details)Jan 15 2020, 4:32 PM
gerritbot added a comment.Jan 15 2020, 4:35 PM

Change 565057 merged by Filippo Giunchedi:
[operations/puppet@production] DHCP: Add MAC address entires for restbase202[1-3]

https://gerrit.wikimedia.org/r/565057

gerritbot added a comment.Jan 15 2020, 4:43 PM

Change 565058 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/puppet@production] Partman: Add restbase202[1-3] to netboot.cfg

https://gerrit.wikimedia.org/r/565058

gerritbot added a comment.Jan 15 2020, 4:52 PM

Change 565058 merged by Filippo Giunchedi:
[operations/puppet@production] Partman: Add restbase202[1-3] to netboot.cfg

https://gerrit.wikimedia.org/r/565058

Papaul added a comment.Jan 15 2020, 4:53 PM
papaul@asw-b-codfw# show | compare 
[edit interfaces interface-range vlan-private1-b-codfw]
     member ge-5/0/1 { ... }
+    member ge-1/0/16;
[edit interfaces interface-range disabled]
-    member ge-1/0/16;
[edit interfaces]
+   ge-1/0/16 {
+       description restbase2021;
+   }
papaul@asw-b-codfw# run show interfaces ge-1/0/16 descriptions 
Interface       Admin Link Description
ge-1/0/16       up    up   restbase2021
papaul@asw-c-codfw# show | compare 
[edit interfaces interface-range vlan-private1-c-codfw]
     member xe-7/0/11 { ... }
+    member ge-1/0/12;
[edit interfaces interface-range disabled]
-    member ge-1/0/12;
[edit interfaces]
+   ge-1/0/12 {
+       description restbase2022;
+   }
apaul@asw-c-codfw> show interfaces ge-1/0/12 descriptions 
Interface       Admin Link Description
ge-1/0/12       up    up   restbase2022
apaul@asw-d-codfw# show | compare 
[edit interfaces interface-range vlan-private1-d-codfw]
     member xe-4/0/15 { ... }
+    member ge-8/0/6;
[edit interfaces interface-range disabled]
-    member ge-8/0/6;
[edit interfaces]
+   ge-8/0/6 {
+       description restbase2023;
+   }
papaul@asw-d-codfw# run show interfaces ge-8/0/6 descriptions 
Interface       Admin Link Description
ge-8/0/6        up    up   restbase2023
Papaul updated the task description. (Show Details)Jan 15 2020, 4:54 PM
Maintenance_bot removed a project: Patch-For-Review.Jan 15 2020, 5:10 PM
Papaul updated the task description. (Show Details)Jan 15 2020, 5:24 PM
Papaul updated the task description. (Show Details)Jan 15 2020, 8:34 PM
Papaul reassigned this task from Papaul to fgiunchedi.Jan 15 2020, 9:32 PM
Papaul updated the task description. (Show Details)
Papaul subscribed.

@fgiunchedi all yours

gerritbot added a comment.Jan 16 2020, 2:29 PM

Change 565281 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] Add restbase2[123] to restbase::production

https://gerrit.wikimedia.org/r/565281

gerritbot added a project: Patch-For-Review.Jan 16 2020, 2:29 PM
gerritbot added a comment.Jan 16 2020, 3:33 PM

Change 565281 merged by Filippo Giunchedi:
[operations/puppet@production] Add restbase2[123] to restbase::production

https://gerrit.wikimedia.org/r/565281

gerritbot added a comment.Jan 16 2020, 3:40 PM

Change 565305 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/dns@master] Add cassandra instances for restbase202[123]

https://gerrit.wikimedia.org/r/565305

gerritbot added a comment.Jan 16 2020, 3:43 PM

Change 565305 merged by Filippo Giunchedi:
[operations/dns@master] Add cassandra instances for restbase202[123]

https://gerrit.wikimedia.org/r/565305

gerritbot added a comment.Jan 16 2020, 3:44 PM

Change 565307 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] hieradata: add restbase202[123] instances

https://gerrit.wikimedia.org/r/565307

gerritbot added a comment.Jan 16 2020, 3:49 PM

Change 565307 merged by Filippo Giunchedi:
[operations/puppet@production] hieradata: add restbase202[123] instances

https://gerrit.wikimedia.org/r/565307

Maintenance_bot removed a project: Patch-For-Review.Jan 16 2020, 4:10 PM
Eevans edited projects, added Platform Team Workboards (Clinic Duty Team); removed Platform Engineering.Jan 16 2020, 6:23 PM
fgiunchedi closed this task as Resolved.Jan 17 2020, 9:15 AM
fgiunchedi reassigned this task from fgiunchedi to Eevans.
fgiunchedi updated the task description. (Show Details)

All done, service is being implemented in T243000

CCicalese_WMF closed subtask T243000: Bootstrap new Cassandra instances: restbase202[123]-{a,b,c} as Resolved.Mar 3 2020, 4:16 PM
• AMooney moved this task from Inbox to Done on the Platform Team Workboards (Clinic Duty Team) board.Mar 24 2021, 5:26 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL