Page MenuHomePhabricator

Rack and setup db1109 and db1110
Closed, ResolvedPublic

Description

This task will track the racking/setup/installation of 2 new databases (db1109 and db1110) for eqiad. These were ordered on T177488.
If possible rack the server in the following locations

db1109: D8
db1110: C3

db1109

  • receive in system on procurement task T177488
  • bios/drac/serial setup/testing
  • raid set up to raid 10 (standard db raid)
  • mgmt dns entries added for both asset tag and hostname
  • production dns entries added
  • network port setup
  • operations/puppet update
  • OS installation
  • puppet/salt accept/initial run
  • handoff for service implementation

db1110

  • receive in system on procurement task T177488
  • bios/drac/serial setup/testing
  • raid set up to raid 10 (standard db raid)
  • mgmt dns entries added for both asset tag and hostname
  • production dns entries added
  • network port setup
  • operations/puppet update
  • OS installation
  • puppet/salt accept/initial run
  • handoff for service implementation

Related Objects

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 16 2017, 3:47 PM

Change 391845 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Allow install db1109 and db1110

https://gerrit.wikimedia.org/r/391845

Change 391845 merged by Marostegui:
[operations/puppet@production] install_server: Allow install db1109 and db1110

https://gerrit.wikimedia.org/r/391845

Cmjohnson updated the task description. (Show Details)Nov 16 2017, 6:58 PM

Change 391879 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/dns@master] Adding mgmt dns for db1109 and 1110 T180700

https://gerrit.wikimedia.org/r/391879

Change 391879 merged by Cmjohnson:
[operations/dns@master] Adding mgmt dns for db1109 and 1110 T180700

https://gerrit.wikimedia.org/r/391879

Marostegui moved this task from Triage to In progress on the DBA board.Nov 17 2017, 6:36 AM

Change 392038 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/puppet@production] adding dhcpd entries for db1109-10 T180700

https://gerrit.wikimedia.org/r/392038

Change 392038 merged by Cmjohnson:
[operations/puppet@production] adding dhcpd entries for db1109-10 T180700

https://gerrit.wikimedia.org/r/392038

Cmjohnson updated the task description. (Show Details)Nov 17 2017, 1:10 PM
Cmjohnson updated the task description. (Show Details)

@Marostegui These are ready for you

Cmjohnson moved this task from Backlog to Blocked on the ops-eqiad board.Nov 17 2017, 2:14 PM

Thanks a lot @Cmjohnson for getting them up and running so fast! Really appreciated!

[14:15:14] marostegui@db1109:~$ uptime
 14:15:15 up 12 min,  1 user,  load average: 0.04, 0.16, 0.09

[14:15:44] marostegui@db1110:~$ uptime
 14:15:50 up 20 min,  1 user,  load average: 0.02, 0.16, 0.09

Change 392068 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Add db1109 and db1110 to s5/s8

https://gerrit.wikimedia.org/r/392068

Change 392068 merged by Marostegui:
[operations/puppet@production] mariadb: Add db1109 and db1110 to s5/s8

https://gerrit.wikimedia.org/r/392068

Mentioned in SAL (#wikimedia-operations) [2017-11-20T06:41:02Z] <marostegui> Stop MySQL on db1082 to clone db1109 and db1110 - T180700

Change 392372 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/software@master] s5.hosts: Add db1109 and db1110 to s5

https://gerrit.wikimedia.org/r/392372

Change 392372 merged by jenkins-bot:
[operations/software@master] s5.hosts: Add db1109 and db1110 to s5

https://gerrit.wikimedia.org/r/392372

Change 392403 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] install_server: Install db1109, db1110 as jessie

https://gerrit.wikimedia.org/r/392403

Change 392403 merged by Marostegui:
[operations/puppet@production] install_server: Install db1109, db1110 as jessie

https://gerrit.wikimedia.org/r/392403

Script wmf-auto-reimage was launched by marostegui on neodymium.eqiad.wmnet for hosts:

db1110.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/201711201109_marostegui_20805_db1110_eqiad_wmnet.log.

Script wmf-auto-reimage was launched by marostegui on neodymium.eqiad.wmnet for hosts:

db1109.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/201711201110_marostegui_20842_db1109_eqiad_wmnet.log.

Completed auto-reimage of hosts:

['db1110.eqiad.wmnet']

and were ALL successful.

Completed auto-reimage of hosts:

['db1109.eqiad.wmnet']

and were ALL successful.

Mentioned in SAL (#wikimedia-operations) [2017-11-20T16:24:27Z] <marostegui> Add db1109 and db1110 to tendril - T180700

Change 392444 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Add db1109 and db1110

https://gerrit.wikimedia.org/r/392444

Change 392444 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Add db1109 and db1110

https://gerrit.wikimedia.org/r/392444

Mentioned in SAL (#wikimedia-operations) [2017-11-20T16:38:14Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Add db1109 and db1110 to the config depooled - T180700 (duration: 00m 48s)

Mentioned in SAL (#wikimedia-operations) [2017-11-20T16:39:13Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Add db1109 and db1110 to the config depooled - T180700 (duration: 00m 48s)

Change 392579 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Restore original weight for db1082

https://gerrit.wikimedia.org/r/392579

Change 392579 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Restore original weight for db1082

https://gerrit.wikimedia.org/r/392579

Change 392584 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Pool db1109 and db1110

https://gerrit.wikimedia.org/r/392584

Change 392585 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Enable notifications db1109,db1110

https://gerrit.wikimedia.org/r/392585

Change 392585 merged by Marostegui:
[operations/puppet@production] mariadb: Enable notifications db1109,db1110

https://gerrit.wikimedia.org/r/392585

Change 392584 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Pool db1109 and db1110

https://gerrit.wikimedia.org/r/392584

Mentioned in SAL (#wikimedia-operations) [2017-11-21T07:03:06Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Pool db1109 and db1110 in s5 with small weight - T180700 (duration: 00m 48s)

Change 392587 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Increase weight for db1109, db1110

https://gerrit.wikimedia.org/r/392587

Change 392587 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Increase weight for db1109, db1110

https://gerrit.wikimedia.org/r/392587

Mentioned in SAL (#wikimedia-operations) [2017-11-21T07:36:28Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase weight for db1109 and db1110 - T180700 (duration: 00m 49s)

Change 392589 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Fully poll db1109,db1110

https://gerrit.wikimedia.org/r/392589

Change 392589 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Fully poll db1109,db1110

https://gerrit.wikimedia.org/r/392589

Mentioned in SAL (#wikimedia-operations) [2017-11-21T08:11:40Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Fully pool db1109 and db1110 - T180700 (duration: 00m 48s)

Marostegui closed this task as Resolved.Nov 21 2017, 8:11 AM

These two servers have been fully pooled in s5.