Page MenuHomePhabricator
Create Task
Maniphest T258750

(2020-08-15) rack/setup/install dbprov1003.eqiad.wmnet
Closed, ResolvedPublic
Actions

Description

This task will track the racking, setup, and OS installation of dbprov1003.eqiad.wmnet

Hostname / Racking / Installation Details

Hostnames: dbprov1003
Racking Proposal: Should not share rows with existing hosts on A7 & B7 for redundancy. 10G required. If not possible, it should not share the same racks.
Networking/Subnet/VLAN/IP: Production network (where production mysql lives), 10G required for high transfer/backup restore to mysql servers.
Partitioning/Raid: Raid setup: Raid6 HDDs (create first so it is the first virtual drive-sda), Raid0 SSDs (create second so it is the second virtual drive-sdb). Both same options as the dbs- write back with 256K of stripe
OS Distro: Buster

Per host setup checklist

dbprov1003.eqiad.wmnet:

  • - receive in system on procurement task T257547 & in coupa
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run (with role:spare)
  • - host state in netbox set to staged

Once the system(s) above have had all checkbox steps completed, this task can be resolved.

Details

SubjectRepoBranchLines +/-
Fixing dbprov1003 site.ppoperations/puppetproduction+3 -3
Updating site.pp file for dbprov1003 switched it to role:insetup to be safeoperations/puppetproduction+1 -1
Adding dbprov1003 mac address to dhcp fileoperations/puppetproduction+5 -0
Adding production dns for dbprov1003operations/dnsmaster+2 -0
mariadb-backups: Add dbprov[12]003 to the db.cfg partman recipe listoperations/puppetproduction+1 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

RobH created this task.Jul 23 2020, 8:20 PM
Restricted Application added a project: SRE. · View Herald TranscriptJul 23 2020, 8:20 PM
RobH renamed this task from (<enter due date here>) rack/setup/install dbprov1003.eqiad.wmnet to (2020-09-30) rack/setup/install dbprov1003.eqiad.wmnet.Jul 23 2020, 8:20 PM
RobH added a parent task: Unknown Object (Task).
RobH unsubscribed.
RobH renamed this task from (2020-09-30) rack/setup/install dbprov1003.eqiad.wmnet to (2020-09-14) rack/setup/install dbprov1003.eqiad.wmnet.Jul 23 2020, 8:22 PM
RobH moved this task from Backlog to Racking Tasks on the ops-eqiad board.Jul 23 2020, 8:27 PM
RobH moved this task from Triage to Blocked external/Not db team on the DBA board.
RobH moved this task from Backlog to Acknowledged on the SRE board.
jcrespo updated the task description. (Show Details)Aug 5 2020, 3:53 PM
gerritbot added a comment.Aug 5 2020, 4:41 PM

Change 618555 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb-backups: Add dbprov[12]003 to the db.cfg partman recipe list

https://gerrit.wikimedia.org/r/618555

gerritbot added a project: Patch-For-Review.Aug 5 2020, 4:41 PM

Change 618555 merged by Jcrespo:
[operations/puppet@production] mariadb-backups: Add dbprov[12]003 to the db.cfg partman recipe list

https://gerrit.wikimedia.org/r/618555

Maintenance_bot removed a project: Patch-For-Review.Aug 5 2020, 5:10 PM
wiki_willy renamed this task from (2020-09-14) rack/setup/install dbprov1003.eqiad.wmnet to (2020-08-15) rack/setup/install dbprov1003.eqiad.wmnet.Aug 10 2020, 4:17 PM
wiki_willy assigned this task to • Cmjohnson.
• Cmjohnson updated the task description. (Show Details)Aug 21 2020, 5:46 PM
gerritbot added a comment.Aug 23 2020, 7:26 PM

Change 621959 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/dns@master] Adding production dns for dbprov1003

https://gerrit.wikimedia.org/r/621959

gerritbot added a project: Patch-For-Review.Aug 23 2020, 7:26 PM
gerritbot added a comment.Aug 23 2020, 7:31 PM

Change 621959 merged by Cmjohnson:
[operations/dns@master] Adding production dns for dbprov1003

https://gerrit.wikimedia.org/r/621959

gerritbot added a comment.Aug 23 2020, 7:36 PM

Change 621960 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/puppet@production] Adding dbprov1003 mac address to dhcp file

https://gerrit.wikimedia.org/r/621960

gerritbot added a comment.Aug 23 2020, 7:37 PM

Change 621960 merged by Cmjohnson:
[operations/puppet@production] Adding dbprov1003 mac address to dhcp file

https://gerrit.wikimedia.org/r/621960

gerritbot added a comment.Aug 23 2020, 7:39 PM

Change 621961 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/puppet@production] Updating site.pp file for dbprov1003 switched it to role:insetup to be safe

https://gerrit.wikimedia.org/r/621961

gerritbot added a comment.Aug 23 2020, 7:41 PM

Change 621961 merged by Cmjohnson:
[operations/puppet@production] Updating site.pp file for dbprov1003 switched it to role:insetup to be safe

https://gerrit.wikimedia.org/r/621961

ops-monitoring-bot added a comment.Aug 23 2020, 7:44 PM

Script wmf-auto-reimage was launched by cmjohnson on cumin1001.eqiad.wmnet for hosts:

dbprov1003.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202008231944_cmjohnson_3546_dbprov1003_eqiad_wmnet.log.

ops-monitoring-bot added a comment.Aug 23 2020, 7:58 PM

Completed auto-reimage of hosts:

['dbprov1003.eqiad.wmnet']

Of which those FAILED:

['dbprov1003.eqiad.wmnet']
ops-monitoring-bot added a comment.Aug 23 2020, 8:00 PM

Script wmf-auto-reimage was launched by cmjohnson on cumin1001.eqiad.wmnet for hosts:

dbprov1003.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202008232000_cmjohnson_5915_dbprov1003_eqiad_wmnet.log.

Maintenance_bot removed a project: Patch-For-Review.Aug 23 2020, 8:10 PM
ops-monitoring-bot added a comment.Aug 23 2020, 8:13 PM

Completed auto-reimage of hosts:

['dbprov1003.eqiad.wmnet']

Of which those FAILED:

['dbprov1003.eqiad.wmnet']
gerritbot added a comment.Aug 23 2020, 8:21 PM

Change 621964 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/puppet@production] Fixing dbprov1003 site.pp

https://gerrit.wikimedia.org/r/621964

gerritbot added a project: Patch-For-Review.Aug 23 2020, 8:21 PM

Change 621964 merged by Cmjohnson:
[operations/puppet@production] Fixing dbprov1003 site.pp

https://gerrit.wikimedia.org/r/621964

ops-monitoring-bot added a comment.Aug 23 2020, 8:22 PM

Script wmf-auto-reimage was launched by cmjohnson on cumin1001.eqiad.wmnet for hosts:

dbprov1003.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/202008232022_cmjohnson_10221_dbprov1003_eqiad_wmnet.log.

• Cmjohnson updated the task description. (Show Details)Aug 23 2020, 8:40 PM
ops-monitoring-bot added a comment.Aug 23 2020, 8:58 PM

Completed auto-reimage of hosts:

['dbprov1003.eqiad.wmnet']

and were ALL successful.

• Cmjohnson closed this task as Resolved.Aug 23 2020, 8:59 PM
• Cmjohnson added a subscriber: jcrespo.

@jcrespo This server is ready for you, I did update the site.pp role to insetup. I didn't want to install it right into the mariadb role. Resolving this task, the on-site portion has been completed.

Maintenance_bot removed a project: Patch-For-Review.Aug 23 2020, 9:10 PM
jcrespo added a comment.Aug 24 2020, 8:08 AM
In T258750#6404800, @Cmjohnson wrote:

@jcrespo This server is ready for you, I did update the site.pp role to insetup. I didn't want to install it right into the mariadb role. Resolving this task, the on-site portion has been completed.

Thanks, Chris, this unblocks us a lot! This is exactly what I would have wanted. Thank you again!

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL