Page MenuHomePhabricator
Create Task
Maniphest T229251

(2019-08-31)rack/setup/install db2131.codfw.wmnet
Closed, ResolvedPublic0 Estimated Story Points
Actions

Description

This task will track the racking, setup, and installation of db2131.codfw.wmnet, ordered via T228201 on 2019-07-22.

Hostnames: db2131.codfw.wmnet
Racking Proposal: Anywhere on row A or D.
Networking/Subnet/VLAN/IP: 1G, normal VLAN for DBs.
Partitioning/Raid: RAID10 + 256KB stripsize with writeback

Need by / Completion Date: 2019-08-31

db2131: ge-8/0/5

  • - receive in system on procurement task T228201
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/hw raid setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, internal vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged
  • - handoff for service implementation
  • - service implementer changes from 'staged' status to 'active' status in netbox'

Details

SubjectRepoBranchLines +/-
DHCP: Add MAC address for db2131operations/puppetproduction+5 -0
DNS: Add mgmt and production DNS for db2131operations/dnsmaster+7 -0
mariadb: Allow reimage of db2131operations/puppetproduction+2 -2
Customize query in gerrit

Related Objects
Search...

Event Timeline

RobH triaged this task as Medium priority.Jul 29 2019, 4:51 PM
RobH created this task.
Restricted Application added a project: SRE. · View Herald TranscriptJul 29 2019, 4:51 PM
RobH renamed this task from rack/setup/install db2131.codfw.wmnet to (2019-08-31)rack/setup/install db2131.codfw.wmnet.Jul 29 2019, 4:51 PM
RobH added a parent task: Unknown Object (Task).
RobH updated the task description. (Show Details)
RobH updated the task description. (Show Details)Jul 29 2019, 4:53 PM
RobH updated the task description. (Show Details)
gerritbot added a comment.Jul 30 2019, 7:39 AM

Change 526378 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Allow reimage of db2131

https://gerrit.wikimedia.org/r/526378

gerritbot added a project: Patch-For-Review.Jul 30 2019, 7:39 AM

Change 526378 merged by Marostegui:
[operations/puppet@production] mariadb: Allow reimage of db2131

https://gerrit.wikimedia.org/r/526378

Marostegui moved this task from Triage to Blocked external/Not db team on the DBA board.Jul 30 2019, 7:46 AM

@RobH @Papaul I have merged: https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/526378/
The only changes pending from your side to be able to install these hosts once they arrive would be:

  • Production DNS entries
  • MGMT DNS entries
  • MAC entries for the DHCP
Maintenance_bot removed a project: Patch-For-Review.Jul 30 2019, 8:10 AM
wiki_willy edited projects, added ops-codfw; removed ops-eqiad.Jul 31 2019, 7:53 AM
Papaul updated the task description. (Show Details)Jul 31 2019, 4:37 PM
Papaul moved this task from Backlog to Racking Tasks on the ops-codfw board.Jul 31 2019, 7:57 PM
Papaul updated the task description. (Show Details)Aug 5 2019, 4:26 PM
Papaul updated the task description. (Show Details)Aug 5 2019, 8:33 PM
gerritbot added a comment.Aug 5 2019, 10:45 PM

Change 528281 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/dns@master] DNS: Add mgmt and production DNS for db2131

https://gerrit.wikimedia.org/r/528281

gerritbot added a project: Patch-For-Review.Aug 5 2019, 10:45 PM
gerritbot added a comment.Aug 5 2019, 10:51 PM

Change 528281 merged by Dzahn:
[operations/dns@master] DNS: Add mgmt and production DNS for db2131

https://gerrit.wikimedia.org/r/528281

gerritbot added a comment.Aug 5 2019, 10:56 PM

Change 528283 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/puppet@production] DHCP: Add MAC address for db2131

https://gerrit.wikimedia.org/r/528283

gerritbot added a comment.Aug 5 2019, 10:58 PM

Change 528283 merged by Dzahn:
[operations/puppet@production] DHCP: Add MAC address for db2131

https://gerrit.wikimedia.org/r/528283

Papaul updated the task description. (Show Details)Aug 5 2019, 11:02 PM
Maintenance_bot removed a project: Patch-For-Review.Aug 5 2019, 11:10 PM
Papaul updated the task description. (Show Details)Aug 5 2019, 11:19 PM
Papaul reassigned this task from Papaul to Marostegui.Aug 5 2019, 11:43 PM
Papaul updated the task description. (Show Details)
Papaul subscribed.

@Marostegui All yours

Marostegui reassigned this task from Marostegui to Papaul.Aug 6 2019, 5:14 AM

Looks like one of the PS isn't giving power:

-------------------------------------------------------------------------------
Record:      2
Date/Time:   08/05/2019 10:42:34
Source:      system
Severity:    Critical
Description: Power supply redundancy is lost.
-------------------------------------------------------------------------------
Record:      3
Date/Time:   08/05/2019 10:42:37
Source:      system
Severity:    Critical
Description: The power input for power supply 1 is lost.
-------------------------------------------------------------------------------

Can you double check it?
Thanks!

Marostegui added a subtask: T228969: Productionize db21[21-31}.Aug 6 2019, 5:17 AM
Papaul reassigned this task from Papaul to Marostegui.Aug 6 2019, 3:00 PM

Fixed

Marostegui added a comment.Aug 7 2019, 5:36 AM

Thanks!
The alert cleared:

-------------------------------------------------------------------------------
Record:      4
Date/Time:   08/06/2019 14:53:16
Source:      system
Severity:    Ok
Description: The input power for power supply 1 has been restored.
-------------------------------------------------------------------------------
Record:      5
Date/Time:   08/06/2019 14:53:18
Source:      system
Severity:    Ok
Description: The power supplies are redundant.
-------------------------------------------------------------------------------
Marostegui closed this task as Resolved.Aug 7 2019, 5:37 AM
Marostegui updated the task description. (Show Details)
Marostegui closed subtask T228969: Productionize db21[21-31} as Resolved.Aug 14 2019, 7:32 AM
• JAufrecht mentioned this in T246037: Document DC-Ops Management Performance reporting needs & design.Feb 24 2020, 8:43 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL