Page MenuHomePhabricator
Create Task
Maniphest T219463

rack/setup/install (5) codfw dedicated dump slaves
Closed, ResolvedPublic
Actions

Description

This task will track the racking, setup, and installation of (5) dedicated dump slaves in codfw.

Racking Proposal: One server per row, and as we have to repeat a row, let's not put them on the same rack on the same row.
RAID: RAID10 with 256kb stripe size

db2097.codfw.wmnet: Row A rack A6

  • - receive in system on procurement task T214069
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - handoff for service implementation T220572

db2098.codfw.wmnet: Row B rack B6

  • - receive in system on procurement task T214069
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - handoff for service implementation T220572

db2099.codfw.wmnet: Row C rack C6

  • - receive in system on procurement task T214069
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - handoff for service implementation T220572

db2100.codfw.wmnet: Row D rack D1

  • - receive in system on procurement task T214069
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - handoff for service implementation T220572

db2101.codfw.wmnet: Row D rack D6

  • - receive in system on procurement task T214069
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - handoff for service implementation T220572

Details

ProjectBranchLines +/-Subject
operations/puppetproduction+5 -0db2[097|098|099|100|101]: Disable notifications
operations/puppetproduction+30 -0DHCP: Add MAC address entries for db209[7-9] and db210[0-2]
operations/dnsmaster+39 -1DNS: Add mgmt and prodcution DNS for db209[7-9] db210[0-2]
operations/puppetproduction+7 -2install_server: Allow image of the new codfw DBs.
Customize query in gerrit

Related Objects
Search...

Event Timeline

RobH triaged this task as Medium priority.Mar 27 2019, 10:04 PM
RobH created this task.
RobH created this object in space Restricted Space.
Restricted Application added a project: procurement. · View Herald TranscriptMar 27 2019, 10:04 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
RobH shifted this object from the Restricted Space space to the S1 Public space.Mar 27 2019, 10:05 PM

Any member of DBA team can provide feedback (@jcrespo or @Marostegui) and please then assign to @Papaul for followup.

• Marostegui reassigned this task from jcrespo to Papaul.Mar 28 2019, 5:47 AM
• Marostegui updated the task description. (Show Details)

Hostnames updated.
Racking proposal, is basically one server per row. And as we have 5 servers and 4 rows, we have to place 2 server within the same row, let's not place them on the same rack for that specific row.
Other than that, we don't mind the racks really.

• Marostegui moved this task from Triage to Blocked external/Not db team on the DBA board.Mar 28 2019, 5:48 AM
gerritbot added a comment.Mar 28 2019, 6:01 AM

Change 499706 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Allow image of the new codfw DBs.

https://gerrit.wikimedia.org/r/499706

gerritbot added a project: Patch-For-Review.Mar 28 2019, 6:01 AM
RobH removed a project: procurement.Mar 28 2019, 6:03 AM
gerritbot added a comment.Mar 28 2019, 7:15 AM

Change 499706 merged by Marostegui:
[operations/puppet@production] install_server: Allow image of the new codfw DBs.

https://gerrit.wikimedia.org/r/499706

• Marostegui updated the task description. (Show Details)Mar 28 2019, 8:08 AM
Papaul updated the task description. (Show Details)Mar 28 2019, 2:27 PM
Papaul added a comment.Mar 28 2019, 2:47 PM

switch port information
db2097
asw-a6-codfw ge-6/0 6

db2098
asw-b6-codfw ge-6/0/0

db2099
asw-c6-codfw ge-6/0/6

db2100
asw-d1-codfw ge-1/0/0

db2101
asw-d6-codfw ge-6/0/12

jcrespo added a parent task: T220002: Decommission dbstore2001.codfw.wmnet and dbstore2002.codfw.wmnet.Apr 3 2019, 3:22 PM
jcrespo renamed this task from rack/setup/install (5) dedicated dump slaves to rack/setup/install (5) codfw dedicated dump slaves.Apr 8 2019, 4:45 PM
Papaul updated the task description. (Show Details)Apr 9 2019, 8:44 PM
gerritbot added a comment.Apr 10 2019, 12:05 AM

Change 502651 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/dns@master] DNS: Add mgmt and prodcution DNS for db209[7-9] db210[0-2]

https://gerrit.wikimedia.org/r/502651

Papaul updated the task description. (Show Details)Apr 10 2019, 12:08 AM
• Marostegui added a parent task: T220572: Productionize eqiad and codfw source backup hosts & codfw backup test host.Apr 10 2019, 4:54 AM
jcrespo mentioned this in T219461: rack/setup/install db2102.codfw.wmnet as a testing host for codfw backups.Apr 10 2019, 6:47 AM
gerritbot added a comment.Apr 10 2019, 4:04 PM

Change 502824 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/puppet@production] DHCP: Add MAC address entries for db209[7-9] and db210[0-2]

https://gerrit.wikimedia.org/r/502824

gerritbot added a comment.Apr 10 2019, 5:20 PM

Change 502651 merged by Marostegui:
[operations/dns@master] DNS: Add mgmt and prodcution DNS for db209[7-9] db210[0-2]

https://gerrit.wikimedia.org/r/502651

gerritbot added a comment.Apr 10 2019, 5:24 PM

Change 502824 merged by Marostegui:
[operations/puppet@production] DHCP: Add MAC address entries for db209[7-9] and db210[0-2]

https://gerrit.wikimedia.org/r/502824

Papaul updated the task description. (Show Details)Apr 11 2019, 2:28 PM
Papaul updated the task description. (Show Details)Apr 11 2019, 2:40 PM
Papaul updated the task description. (Show Details)Apr 11 2019, 3:15 PM
Papaul updated the task description. (Show Details)Apr 11 2019, 3:32 PM
Papaul updated the task description. (Show Details)Apr 11 2019, 3:46 PM
Papaul updated the task description. (Show Details)Apr 11 2019, 4:28 PM
Papaul added a comment.Apr 11 2019, 4:37 PM

Putting this here for reference

debian-installer

For some reason, and I heard some rumours that this is a known bug, I had to disable USB support and the SD-card reader in the interface formerly known as BIOS. Otherwise the installer detects the first volume of the P408i raid controller as "/dev/sdb" instead of "/dev/sda".

https://sven.stormbind.net/blog/posts/deb_stretch_hpe_dl360/

• Marostegui closed this task as Resolved.Apr 12 2019, 5:01 AM
• Marostegui updated the task description. (Show Details)

All these host are now ready to be productionized at T220572.
There is a problem with the controller exposure to the OS which is being discussed at that same task (T220572#5104585)
Thanks @Papaul for being so fast with these and for helping out investigating the controller issues on that other task!

gerritbot added a comment.Apr 12 2019, 5:07 AM

Change 503164 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db2[097|098|099|100|101]: Disable notifications

https://gerrit.wikimedia.org/r/503164

gerritbot added a comment.Apr 12 2019, 5:10 AM

Change 503164 merged by Marostegui:
[operations/puppet@production] db2[097|098|099|100|101]: Disable notifications

https://gerrit.wikimedia.org/r/503164

Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL