Page MenuHomePhabricator

(2019-09-15) rack/setup/install ms-be105[1-6].eqiad.wmnet
Open, HighPublic

Description

This task will track the setup and installation of 6 new ms-be hosts ordered via T228461.

Hostnames: ms-be105[1-6].eqiad.wmnet
Racking Proposal: 10G racks, 2x hosts in rows A+C+D and 3x in row B to refresh old hw. Rack diversity with existing ms-be hw is preferred but not required.
Networking/Subnet/VLAN/IP: Internal subnet per row
Partitioning/Raid: use existing ms-be setup

ms-be1051:

  • - receive in system on procurement task T228461
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

ms-be1052:

  • - receive in system on procurement task T228461
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

ms-be1053:

  • - receive in system on procurement task T228461
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

ms-be1054:

  • - receive in system on procurement task T228461
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

ms-be1055:

  • - receive in system on procurement task T228461
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

ms-be1056:

  • - receive in system on procurement task T228461
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

Event Timeline

RobH triaged this task as Normal priority.Sep 9 2019, 5:02 PM
RobH created this task.
Restricted Application added a project: Operations. · View Herald TranscriptSep 9 2019, 5:02 PM
RobH renamed this task from rack/setup/install ms-be105[1-6].eqiad.wmnet to (2019-09-15) rack/setup/install ms-be105[1-6].eqiad.wmnet.Sep 9 2019, 5:02 PM
RobH added a parent task: Unknown Object (Task).
RobH edited parent tasks, added: Unknown Object (Task); removed: Unknown Object (Task).

I've set the due date via the require by date on the ordering task:

Need By Date

Mid Q1, to get enough time to be fully in service before EOQ

wiki_willy moved this task from Backlog to Racking Tasks on the ops-eqiad board.Sep 9 2019, 5:05 PM

Change 537171 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/dns@master] Adding mgmt dns for new ms-be servers

https://gerrit.wikimedia.org/r/537171

Change 537171 merged by Cmjohnson:
[operations/dns@master] Adding mgmt dns for new ms-be servers

https://gerrit.wikimedia.org/r/537171

Cmjohnson updated the task description. (Show Details)Thu, Sep 26, 5:39 PM

@fgiunchedi I see you said for raid Partitioning/Raid: "use existing ms-be setup" Unfortunately my memory is not that great anymore can you please remind what the existing raid setup is please,

@fgiunchedi I see you said for raid Partitioning/Raid: "use existing ms-be setup" Unfortunately my memory is not that great anymore can you please remind what the existing raid setup is please,

For sure! All disks in raid0 including ssd, and the ssd must show up as sda/sdb and boot devices, thanks!

I see this task is for 6x hosts and parent T228461 is for 9x, wanted to make sure that's expected/wanted ?

@fgiunchedi 3 backend systems to replace ms-be101[6-8]

@fgiunchedi 3 backend systems to replace ms-be101[6-8]

Thanks @Jclark-ctr, I wanted to make sure the 3x replacement systems not in this task for rack/install/etc are tracked somewhere

fgiunchedi raised the priority of this task from Normal to High.Wed, Oct 2, 3:43 PM

Please note that putting these systems in production is becoming urgent, is there a status update and/or ETA?

Cmjohnson updated the task description. (Show Details)Tue, Oct 8, 3:50 PM

@fgiunchedi all the on-site work has been completed...they need production DNS

wiki_willy reassigned this task from Jclark-ctr to RobH.Tue, Oct 8, 4:04 PM
wiki_willy added a subscriber: wiki_willy.

@RobH - can you take care of DNS for this to get things completed from the dc-ops side for this install? This one's super urgent, so if you can complete in the AM, it would be much appreciated. Thanks, Willy

Change 541567 had a related patch set uploaded (by RobH; owner: RobH):
[operations/dns@master] ms-be105[1-6] production dns

https://gerrit.wikimedia.org/r/541567

Change 541567 merged by RobH:
[operations/dns@master] ms-be105[1-6] production dns

https://gerrit.wikimedia.org/r/541567

RobH updated the task description. (Show Details)Tue, Oct 8, 5:02 PM
RobH updated the task description. (Show Details)Tue, Oct 8, 5:11 PM

Change 541578 had a related patch set uploaded (by RobH; owner: RobH):
[operations/puppet@production] setting most new ms-be hosts mac addresses

https://gerrit.wikimedia.org/r/541578

Change 541578 merged by RobH:
[operations/puppet@production] setting most new ms-be hosts mac addresses

https://gerrit.wikimedia.org/r/541578

RobH updated the task description. (Show Details)

Change 541579 had a related patch set uploaded (by RobH; owner: RobH):
[operations/puppet@production] fixing mac entries for new ms-be systems

https://gerrit.wikimedia.org/r/541579

Change 541579 merged by RobH:
[operations/puppet@production] fixing mac entries for new ms-be systems

https://gerrit.wikimedia.org/r/541579

Change 541652 had a related patch set uploaded (by RobH; owner: RobH):
[operations/puppet@production] ms-be105[15] dhcp info

https://gerrit.wikimedia.org/r/541652

Change 541652 merged by RobH:
[operations/puppet@production] ms-be105[15] dhcp info

https://gerrit.wikimedia.org/r/541652

RobH updated the task description. (Show Details)Tue, Oct 8, 11:54 PM
RobH updated the task description. (Show Details)Tue, Oct 8, 11:56 PM
RobH updated the task description. (Show Details)Wed, Oct 9, 12:12 AM
RobH updated the task description. (Show Details)Wed, Oct 9, 12:28 AM
RobH added a comment.Wed, Oct 9, 1:38 AM

@fgiunchedi:

the task description has been updated

  • ms-be105[1236] are all yours
  • i need to finish installation/setup of ms-be105[45]
RobH reassigned this task from RobH to fgiunchedi.Wed, Oct 9, 6:30 PM
RobH updated the task description. (Show Details)
RobH removed a project: ops-eqiad.

@fgiunchedi,

ms-be105[1-6].eqiad.wmnet are all online and calling into puppet. You can push them into service as you see fit.

Please note when you push them into service, you still need to change their netbox status from 'staged' to 'active'. You can see the list of the ms-be105* with staged status in netbox here: https://netbox.wikimedia.org/dcim/devices/?q=ms-be105&status=3

Please resolve this task once they are all set to active.

Thanks!

RobH removed a subscriber: RobH.Wed, Oct 9, 6:32 PM

Change 541910 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] hieradata: add ms-be105[1-6]

https://gerrit.wikimedia.org/r/541910

Change 541910 merged by Filippo Giunchedi:
[operations/puppet@production] hieradata: add ms-be105[1-6]

https://gerrit.wikimedia.org/r/541910

Change 541916 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] hieradata: use servers _per_port with ms-be105[1-6]

https://gerrit.wikimedia.org/r/541916

Change 541916 merged by Filippo Giunchedi:
[operations/puppet@production] hieradata: use servers _per_port with ms-be105[1-6]

https://gerrit.wikimedia.org/r/541916

Mentioned in SAL (#wikimedia-operations) [2019-10-09T21:27:17Z] <godog> swift eqiad-prod: add ms-be105[1-6] - T232367

fgiunchedi moved this task from Backlog to Doing on the User-fgiunchedi board.

Mentioned in SAL (#wikimedia-operations) [2019-10-10T19:33:29Z] <godog> swift eqiad-prod: add weight to ms-be105[1-6] - T232367

Mentioned in SAL (#wikimedia-operations) [2019-10-17T09:39:38Z] <godog> swift eqiad-prod: add weight to ms-be105[1-6] - T232367

Mentioned in SAL (#wikimedia-operations) [2019-10-21T08:08:44Z] <godog> swift eqiad-prod: add weight to ms-be105[1-6] - T232367