Page MenuHomePhabricator
Create Task
Maniphest T276644

(Need By: 2021-04-30) rack/setup/install wcqs100[123]
Closed, ResolvedPublic
Actions

Description

This task will track the racking, setup, and OS installation of wcqs100[123]

Hostname / Racking / Installation Details

Hostnames: wcqs1001, wcqs1002, wcqs1003
Racking Proposal: the 3 servers should be in different rows
Networking/Subnet/VLAN/IP: 1G, single port, production VLAN
Partitioning/Raid: S/W RAID10 - partman: raid10-4dev.cfg
OS Distro: Buster

Per host setup checklist

wcqs1001:

  • - receive in system on procurement task T274389 & in coupa
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - add mgmt dns (asset tag and hostname) and production dns entries in netbox, run cookbook sre.dns.netbox.
  • - network port setup via netbox, run homer to commit
  • - bios and idrac firmware updated
  • - operations/puppet update - this should include updates to install_server dhcp and netboot, and site.pp role(insetup) or cp systems use role(insetup::nofirm). - https://gerrit.wikimedia.org/r/682704
  • - OS installation & initital puppet run via wmf-auto-reimage or wmf-auto-reimage-host
  • - host state in netbox set to staged

wcqs1002:

  • - receive in system on procurement task T274389 & in coupa
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - add mgmt dns (asset tag and hostname) and production dns entries in netbox, run cookbook sre.dns.netbox.
  • - network port setup via netbox, run homer to commit
  • - bios and idrac firmware updated
  • - operations/puppet update - this should include updates to install_server dhcp and netboot, and site.pp role(insetup) or cp systems use role(insetup::nofirm). - https://gerrit.wikimedia.org/r/682704
  • - OS installation & initital puppet run via wmf-auto-reimage or wmf-auto-reimage-host
  • - host state in netbox set to staged

wcqs1003:

  • - receive in system on procurement task T274389 & in coupa
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - add mgmt dns (asset tag and hostname) and production dns entries in netbox, run cookbook sre.dns.netbox.
  • - network port setup via netbox, run homer to commit
  • - bios and idrac firmware updated
  • - operations/puppet update - this should include updates to install_server dhcp and netboot, and site.pp role(insetup) or cp systems use role(insetup::nofirm). - https://gerrit.wikimedia.org/r/682704
  • - OS installation & initital puppet run via wmf-auto-reimage or wmf-auto-reimage-host
  • - host state in netbox set to staged

Once the system(s) above have had all checkbox steps completed, this task can be resolved.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
wcqs100[123] setup infooperations/puppetproduction+21 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

RobH created this task.Mar 6 2021, 12:06 AM
RobH assigned this task to Jclark-ctr.
RobH mentioned this in Unknown Object (Task).
RobH added a parent task: Unknown Object (Task).
RobH moved this task from Backlog to Racking Tasks on the ops-eqiad board.
RobH added a project: Discovery-Search (Current work).
RobH unsubscribed.
RobH renamed this task from (Need By: TBD) rack/setup/install wcqs100[123] to (Need By: 2021-04-30) rack/setup/install wcqs100[123].Mar 6 2021, 12:19 AM
Maintenance_bot added a project: SRE.Mar 6 2021, 12:57 AM
Gehel moved this task from Incoming to Blocked/Waiting on the Discovery-Search (Current work) board.Mar 8 2021, 4:49 PM
Jclark-ctr added a comment.EditedApr 12 2021, 9:28 PM

wcqs1001. A3. U37 Port 26 ID3962
wcqs1002 B1. U34 Port 19 ID2950
wcqs1003. C4. U17 Port 15 ID1844

Jclark-ctr reassigned this task from Jclark-ctr to • Cmjohnson.Apr 12 2021, 9:29 PM
Jclark-ctr updated the task description. (Show Details)
Jclark-ctr subscribed.
• Cmjohnson reassigned this task from • Cmjohnson to Jclark-ctr.Apr 15 2021, 7:39 PM
• Cmjohnson subscribed.

@Jclark-ctr netbox script ran for wcqs1001 and 1002. I'm not sure why 1003 is in C4, that's a 10G rack. If it is can you please move to a standard rack please.

ayounsi subscribed.Apr 16 2021, 9:06 AM

@Cmjohnson asw2-a-eqiad and asw2-b-eqiad have outstanding changes, please make sure to commit them.

Changes for 1 devices: ['asw2-a-eqiad.mgmt.eqiad.wmnet']

[edit interfaces interface-range disabled]
-    member ge-3/0/26;
[edit interfaces interface-range vlan-private1-a-eqiad]
     member ge-3/0/24 { ... }
+    member ge-3/0/26;
     member ge-3/0/29 { ... }
[edit interfaces]
+   ge-3/0/26 {
+       description "wcqs1001 {#3962}";
+   }

Changes for 1 devices: ['asw2-b-eqiad.mgmt.eqiad.wmnet']

[edit interfaces interface-range disabled]
-    member ge-1/0/19;
[edit interfaces interface-range vlan-private1-b-eqiad]
     member ge-1/0/16 { ... }
+    member ge-1/0/19;
     member ge-1/0/22 { ... }
[edit interfaces]
+   ge-1/0/19 {
+       description "wcqs1002 {#2950}";
+   }
ayounsi unsubscribed.Apr 16 2021, 9:06 AM
Jclark-ctr added a comment.Apr 16 2021, 6:10 PM

@Cmjohnson my mistake it was C5 for 1003. netbox was correct though

Jclark-ctr reassigned this task from Jclark-ctr to • Cmjohnson.Apr 16 2021, 6:11 PM
Jclark-ctr updated the task description. (Show Details)Apr 16 2021, 6:14 PM
• Cmjohnson reassigned this task from • Cmjohnson to RobH.Apr 23 2021, 5:02 PM
• Cmjohnson updated the task description. (Show Details)
• Cmjohnson added a subscriber: RobH.

Assigning this to @RobH to complete install

gerritbot added a comment.Apr 26 2021, 6:26 PM

Change 682704 had a related patch set uploaded (by RobH; author: RobH):

[operations/puppet@production] wcqs100[123] setup info

https://gerrit.wikimedia.org/r/682704

gerritbot added a project: Patch-For-Review.Apr 26 2021, 6:26 PM

Change 682704 merged by RobH:

[operations/puppet@production] wcqs100[123] setup info

https://gerrit.wikimedia.org/r/682704

ops-monitoring-bot added a comment.Apr 26 2021, 6:30 PM

Script wmf-auto-reimage was launched by robh on cumin1001.eqiad.wmnet for hosts:

['wcqs1001.eqiad.wmnet', 'wcqs1002.eqiad.wmnet', 'wcqs1003.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/202104261830_robh_12667.log.

RobH updated the task description. (Show Details)Apr 26 2021, 6:31 PM
ops-monitoring-bot added a comment.Apr 26 2021, 6:59 PM

Completed auto-reimage of hosts:

['wcqs1001.eqiad.wmnet', 'wcqs1002.eqiad.wmnet', 'wcqs1003.eqiad.wmnet']

and were ALL successful.

RobH closed this task as Resolved.Apr 26 2021, 7:06 PM
RobH removed a project: Patch-For-Review.
RobH updated the task description. (Show Details)
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits