Page MenuHomePhabricator

rack/setup/install (5) codfw dedicated dump slaves
Closed, ResolvedPublic

Description

This task will track the racking, setup, and installation of (5) dedicated dump slaves in codfw.

Racking Proposal: One server per row, and as we have to repeat a row, let's not put them on the same rack on the same row.
RAID: RAID10 with 256kb stripe size

db2097.codfw.wmnet: Row A rack A6

  • - receive in system on procurement task T214069
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - handoff for service implementation T220572

db2098.codfw.wmnet: Row B rack B6

  • - receive in system on procurement task T214069
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - handoff for service implementation T220572

db2099.codfw.wmnet: Row C rack C6

  • - receive in system on procurement task T214069
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - handoff for service implementation T220572

db2100.codfw.wmnet: Row D rack D1

  • - receive in system on procurement task T214069
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - handoff for service implementation T220572

db2101.codfw.wmnet: Row D rack D6

  • - receive in system on procurement task T214069
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - handoff for service implementation T220572

Details

Event Timeline

RobH triaged this task as Medium priority.Mar 27 2019, 10:04 PM
RobH created this task.
RobH created this object in space Restricted Space.
Restricted Application added a project: procurement. · View Herald TranscriptMar 27 2019, 10:04 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
RobH shifted this object from the Restricted Space space to the S1 Public space.Mar 27 2019, 10:05 PM

Any member of DBA team can provide feedback (@jcrespo or @Marostegui) and please then assign to @Papaul for followup.

Marostegui reassigned this task from jcrespo to Papaul.Mar 28 2019, 5:47 AM
Marostegui updated the task description. (Show Details)

Hostnames updated.
Racking proposal, is basically one server per row. And as we have 5 servers and 4 rows, we have to place 2 server within the same row, let's not place them on the same rack for that specific row.
Other than that, we don't mind the racks really.

Change 499706 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Allow image of the new codfw DBs.

https://gerrit.wikimedia.org/r/499706

Change 499706 merged by Marostegui:
[operations/puppet@production] install_server: Allow image of the new codfw DBs.

https://gerrit.wikimedia.org/r/499706

Marostegui updated the task description. (Show Details)Mar 28 2019, 8:08 AM
Papaul updated the task description. (Show Details)Mar 28 2019, 2:27 PM

switch port information
db2097
asw-a6-codfw ge-6/0 6

db2098
asw-b6-codfw ge-6/0/0

db2099
asw-c6-codfw ge-6/0/6

db2100
asw-d1-codfw ge-1/0/0

db2101
asw-d6-codfw ge-6/0/12

jcrespo renamed this task from rack/setup/install (5) dedicated dump slaves to rack/setup/install (5) codfw dedicated dump slaves.Apr 8 2019, 4:45 PM
Papaul updated the task description. (Show Details)Apr 9 2019, 8:44 PM

Change 502651 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/dns@master] DNS: Add mgmt and prodcution DNS for db209[7-9] db210[0-2]

https://gerrit.wikimedia.org/r/502651

Papaul updated the task description. (Show Details)Apr 10 2019, 12:08 AM

Change 502824 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/puppet@production] DHCP: Add MAC address entries for db209[7-9] and db210[0-2]

https://gerrit.wikimedia.org/r/502824

Change 502651 merged by Marostegui:
[operations/dns@master] DNS: Add mgmt and prodcution DNS for db209[7-9] db210[0-2]

https://gerrit.wikimedia.org/r/502651

Change 502824 merged by Marostegui:
[operations/puppet@production] DHCP: Add MAC address entries for db209[7-9] and db210[0-2]

https://gerrit.wikimedia.org/r/502824

Papaul updated the task description. (Show Details)Apr 11 2019, 2:28 PM
Papaul updated the task description. (Show Details)Apr 11 2019, 2:40 PM
Papaul updated the task description. (Show Details)Apr 11 2019, 3:15 PM
Papaul updated the task description. (Show Details)Apr 11 2019, 3:32 PM
Papaul updated the task description. (Show Details)Apr 11 2019, 3:46 PM
Papaul updated the task description. (Show Details)Apr 11 2019, 4:28 PM

Putting this here for reference

debian-installer

For some reason, and I heard some rumours that this is a known bug, I had to disable USB support and the SD-card reader in the interface formerly known as BIOS. Otherwise the installer detects the first volume of the P408i raid controller as "/dev/sdb" instead of "/dev/sda".

https://sven.stormbind.net/blog/posts/deb_stretch_hpe_dl360/

Marostegui closed this task as Resolved.Apr 12 2019, 5:01 AM
Marostegui updated the task description. (Show Details)

All these host are now ready to be productionized at T220572.
There is a problem with the controller exposure to the OS which is being discussed at that same task (T220572#5104585)
Thanks @Papaul for being so fast with these and for helping out investigating the controller issues on that other task!

Change 503164 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db2[097|098|099|100|101]: Disable notifications

https://gerrit.wikimedia.org/r/503164

Change 503164 merged by Marostegui:
[operations/puppet@production] db2[097|098|099|100|101]: Disable notifications

https://gerrit.wikimedia.org/r/503164