Page MenuHomePhabricator

rack/setup/install ganeti10([09]|1[0-8[).eqiad.wmnet
Open, NormalPublic0 Story Points

Description

Please note that we ordered 14 total ganeti nodes via procurement task T214088.

The breakdown is as follows: eqiad: (4) ganeti refresh + (4) ganeti nodes for CI (releng) + (6) eqiad: ganeti nodes for expansion

This breakdown makes it seem like 10 of these will go into the general ganeti pool, and 4 will go for CI/release engineering use.

This task will track the racking and setup of the 10 general ganeti nodes. For this, 4 of them are replacing ganeti100[1-4], while the rest merely expand the service group.

Hostname Proposal: ganeti1009+

Racking Proposal: 4 of these nodes can share with ganeti100[1-4], which are in C4 and C7 (10G racks with 1G hosts, so best to avoid 10G racks for these 1G new hosts as well) We also have existing ganeti nodes in A4, A3 (2), A6, and A5. So place the new nodes as follows: Any 1G rack that isn't A3, A4, A5, or A6. Ideally we spread these out to rows C and D some.

ganeti1009:

  • - receive in system on procurement task T214088
  • - rack system with C3/u28 & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

genati1010:

  • - receive in system on procurement task T214088
  • - rack system C5/u8 & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

genati1011:

  • - receive in system on procurement task T214088
  • - rack system C6/u37 & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

genati1012:

  • - receive in system on procurement task T214088
  • - rack system C8/u20 & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

genati1013:

  • - receive in system on procurement task T214088
  • - rack system B3/u14 & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

genati1014:

  • - receive in system on procurement task T214088
  • - rack system B3/u15 & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

genati1015:

  • - receive in system on procurement task T214088
  • - rack system B5/u21 & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

genati1016:

  • - receive in system on procurement task T214088
  • - rack system B5/u22 & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

genati1017:

  • - receive in system on procurement task T214088
  • - rack system B6/u31& update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

genati1018:

  • - receive in system on procurement task T214088
  • - rack system B8/u29 & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

Event Timeline

RobH triaged this task as Normal priority.Jul 24 2019, 6:59 PM
RobH created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 24 2019, 6:59 PM
RobH assigned this task to akosiaris.Jul 24 2019, 7:00 PM
RobH added subscribers: Cmjohnson, akosiaris.

@akosiaris,

Can i get your sign off about the racking proposal and planning for these 10 ganeti nodes? 4 were refresh, while 6 were expansion from last years budget. If these differ from normal ganeti nodes in any way, please note and assign to @Cmjohnson for racking/followup.

Thanks!

RobH moved this task from Backlog to Racking Tasks on the ops-eqiad board.

@RobH, @Cmjohnson

Indeed the refreshes are for ganeti100[1-4] so row C it is. Try to spread them across 1G racks.

However, the 6 ganeti nodes of the expansion should go either to row B or D, with a preference for B as we want to expand to a 3rd rack row to provide HA for services that need to be spread across an odd number of rows. The "try to spread them across 1G racks" holds true here as well.

The partman recipe should be partman/ganeti-raid5.cfg

Hostnames LGTM, i.e. ganeti1009-ganeti1018

RobH added a parent task: Unknown Object (Task).Jul 30 2019, 5:26 PM
Cmjohnson updated the task description. (Show Details)Aug 13 2019, 2:36 PM
Cmjohnson reassigned this task from akosiaris to Jclark-ctr.Aug 13 2019, 2:41 PM

Please rack, label and cable these servers with the racking locations above. Add them to netbox, be sure to make sure status is set to planned and asset tag/SN is ALL CAPS. Please update the task with which network ports each server is attached to on the access switch.

Change 530132 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/dns@master] Adding mgmt dns ganeti1009-1022

https://gerrit.wikimedia.org/r/530132

Change 530132 merged by Cmjohnson:
[operations/dns@master] Adding mgmt dns ganeti1009-1022

https://gerrit.wikimedia.org/r/530132

@Jclark-ctr Mgmt IP's that need to be setup on the idrac

Instructions for setup https://wikitech.wikimedia.org/wiki/Platform-specific_documentation/Dell_PowerEdge_RN30#Initial_System_Setup

ganeti1009 10.65.5.104
ganeti1010 10.65.5.105
ganeti1011 . 10.65.5.106
ganeti1012 10.65.5.107
ganeti1013 10.65.5.108
ganeti1014 10.65.5.109
ganeti1015 10.65.5.110
ganeti1016 10.65.5.111
ganeti1017 10.65.5.112
ganeti1018 10.65.5.113

Jclark-ctr updated the task description. (Show Details)Aug 14 2019, 1:46 PM

entered ip addresses in IDRAC and set password ganeti10([09]|1[0-8[)

Jclark-ctr added a comment.EditedWed, Sep 4, 10:12 PM

@Cmjohnson Idrac and bios settings finished

host_name port
ganeti1009 23
ganeti1010 7
ganeti1011 41
ganeti1012 1
ganeti1013 14
ganeti1014 15
ganeti1015 28
ganeti1016 29
ganeti1017 30
ganeti1018 28