Page MenuHomePhabricator
Create Task
Maniphest T184161

Productionize 2 new eqiad database servers
Closed, ResolvedPublic
Actions

Description

More than a task, this is a tracking list to not "lose" not yet in-production servers and coordinate how to set them up.

State:

  • db1113: pre-provisioned as db1113:3315 and db1113:3316 (not closing this task until they are fully provisioned in mediawiki)
  • db1114: provisioned in s1

Details

ProjectBranchLines +/-Subject
operations/mediawiki-configmaster+4 -3db-eqiad.php: Pool db1113:3316
operations/mediawiki-configmaster+4 -3db-eqiad.php: Pool db1113:3315
operations/mediawiki-configmaster+6 -6db-eqiad.php: Depool db1063
operations/mediawiki-configmaster+6 -6db-eqiad.php: Depool db1051
operations/mediawiki-configmaster+4 -0db-eqiad,db-codfw.php: Add initial config db1113
operations/softwaremaster+2 -0s5,s6.hosts: Add db1113
operations/puppetproduction+11 -12mariadb: Initial setup for db1113
operations/puppetproduction+10 -0site.pp: Add db111{3,4} to spare
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
Unknown Object (Task)
 Resolved• MarosteguiT182896 Rack and setup db1113 and db1114
 Resolved• MarosteguiT184161 Productionize 2 new eqiad database servers

Event Timeline

• Marostegui triaged this task as Medium priority.Jan 4 2018, 8:20 AM
• Marostegui created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 4 2018, 8:20 AM
• Marostegui moved this task from Triage to Pending comment on the DBA board.Jan 4 2018, 8:20 AM
jcrespo added a comment.Jan 4 2018, 8:21 AM

Maybe solve T182896 if we leave this open?

gerritbot added a subscriber: gerritbot.Jan 4 2018, 8:22 AM

Change 401905 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] site.pp: Add db111{3,4} to spare

https://gerrit.wikimedia.org/r/401905

gerritbot added a project: Patch-For-Review.Jan 4 2018, 8:22 AM
• Marostegui added a comment.Jan 4 2018, 8:23 AM
In T184161#3874432, @jcrespo wrote:

Maybe solve T182896 if we leave this open?

hehe yeah, I was about to :)

gerritbot added a comment.Jan 4 2018, 8:31 AM

Change 401905 merged by Marostegui:
[operations/puppet@production] site.pp: Add db111{3,4} to spare

https://gerrit.wikimedia.org/r/401905

• Marostegui removed • Marostegui as the assignee of this task.Feb 9 2018, 12:18 PM
• Marostegui updated the task description. (Show Details)Mar 9 2018, 10:29 AM
gerritbot added a comment.Mar 9 2018, 10:35 AM

Change 417781 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Provision db1113

https://gerrit.wikimedia.org/r/417781

gerritbot added a comment.Mar 9 2018, 10:47 AM

Change 417781 merged by Marostegui:
[operations/puppet@production] mariadb: Initial setup for db1113

https://gerrit.wikimedia.org/r/417781

ops-monitoring-bot added a subscriber: ops-monitoring-bot.Mar 9 2018, 10:49 AM

Script wmf-auto-reimage was launched by marostegui on neodymium.eqiad.wmnet for hosts:

db1113.eqiad.wmnet

The log can be found in /var/log/wmf-auto-reimage/201803091048_marostegui_30692_db1113_eqiad_wmnet.log.

gerritbot added a comment.Mar 9 2018, 11:05 AM

Change 417796 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/software@master] s5,s6.hosts: Add db1113

https://gerrit.wikimedia.org/r/417796

ops-monitoring-bot added a comment.Mar 9 2018, 11:06 AM

Completed auto-reimage of hosts:

['db1113.eqiad.wmnet']

and were ALL successful.

gerritbot added a comment.Mar 9 2018, 11:08 AM

Change 417796 merged by jenkins-bot:
[operations/software@master] s5,s6.hosts: Add db1113

https://gerrit.wikimedia.org/r/417796

gerritbot added a comment.Mar 9 2018, 11:10 AM

Change 417799 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1051

https://gerrit.wikimedia.org/r/417799

gerritbot added a comment.Mar 9 2018, 11:13 AM

Change 417799 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1051

https://gerrit.wikimedia.org/r/417799

Stashbot added a subscriber: Stashbot.Mar 9 2018, 11:14 AM

Mentioned in SAL (#wikimedia-operations) [2018-03-09T11:14:47Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1051 to clone db1113 - T184161 (duration: 00m 58s)

Stashbot added a comment.Mar 9 2018, 11:15 AM

Mentioned in SAL (#wikimedia-operations) [2018-03-09T11:15:02Z] <marostegui> Stop MySQL on db1051 to clone db1113 - https://phabricator.wikimedia.org/T184161

gerritbot added a comment.Mar 9 2018, 11:28 AM

Change 417812 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Add initial config db1113

https://gerrit.wikimedia.org/r/417812

gerritbot added a comment.Mar 9 2018, 11:33 AM

Change 417812 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Add initial config db1113

https://gerrit.wikimedia.org/r/417812

Stashbot added a comment.Mar 9 2018, 11:34 AM

Mentioned in SAL (#wikimedia-operations) [2018-03-09T11:34:48Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Add initial config for db1113 multiinstance - T184161 (duration: 00m 58s)

• Marostegui claimed this task.Mar 9 2018, 11:35 AM
• Marostegui moved this task from Pending comment to In progress on the DBA board.
Stashbot added a comment.Mar 9 2018, 11:35 AM

Mentioned in SAL (#wikimedia-operations) [2018-03-09T11:35:51Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Add initial config for db1113 multiinstance - T184161 (duration: 00m 58s)

Stashbot added a comment.Mar 9 2018, 12:06 PM

Mentioned in SAL (#wikimedia-operations) [2018-03-09T12:06:58Z] <marostegui@tin> Synchronized wmf-config/db-codfw.php: Repool db1051 after cloning db1113:3315- T184161 (duration: 00m 58s)

Stashbot added a comment.Mar 9 2018, 12:08 PM

Mentioned in SAL (#wikimedia-operations) [2018-03-09T12:08:02Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1051 after cloning db1113:3315- T184161 (duration: 00m 58s)

gerritbot added a comment.Mar 9 2018, 12:11 PM

Change 417832 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Depool db1063

https://gerrit.wikimedia.org/r/417832

gerritbot added a comment.Mar 9 2018, 12:15 PM

Change 417832 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Depool db1063

https://gerrit.wikimedia.org/r/417832

Stashbot added a comment.Mar 9 2018, 12:16 PM

Mentioned in SAL (#wikimedia-operations) [2018-03-09T12:16:39Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1063 to clone db1113:3316 - T184161 (duration: 00m 58s)

Stashbot added a comment.Mar 9 2018, 12:16 PM

Mentioned in SAL (#wikimedia-operations) [2018-03-09T12:16:56Z] <marostegui> Stop mysql on db1063 to clone db1113:3316 - T184161

Stashbot added a comment.Mar 9 2018, 12:26 PM

Mentioned in SAL (#wikimedia-operations) [2018-03-09T12:26:14Z] <marostegui> Compress s5 on db1113:3315 - T184161

Stashbot added a comment.Mar 9 2018, 1:12 PM

Mentioned in SAL (#wikimedia-operations) [2018-03-09T13:12:19Z] <marostegui> Compress s6 on db1113:3316 - T184161

• Marostegui updated the task description. (Show Details)Mar 9 2018, 1:17 PM
Stashbot added a comment.Mar 9 2018, 3:20 PM

Mentioned in SAL (#wikimedia-operations) [2018-03-09T15:20:23Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1063 after cloning db1113:3316 - T184161 (duration: 00m 58s)

gerritbot added a comment.Mar 12 2018, 6:47 AM

Change 418847 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Pool db1113:3315

https://gerrit.wikimedia.org/r/418847

gerritbot added a comment.Mar 12 2018, 6:50 AM

Change 418847 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Pool db1113:3315

https://gerrit.wikimedia.org/r/418847

Stashbot added a comment.Mar 12 2018, 6:54 AM

Mentioned in SAL (#wikimedia-operations) [2018-03-12T06:54:37Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Pool db1113:3315 as vslow,dump in s5 - T184161 (duration: 00m 58s)

gerritbot added a comment.Mar 12 2018, 7:04 AM

Change 418848 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad.php: Pool db1113:3316

https://gerrit.wikimedia.org/r/418848

gerritbot added a comment.Mar 12 2018, 7:09 AM

Change 418848 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad.php: Pool db1113:3316

https://gerrit.wikimedia.org/r/418848

Stashbot added a comment.Mar 12 2018, 7:10 AM

Mentioned in SAL (#wikimedia-operations) [2018-03-12T07:10:54Z] <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Pool db1113:3316 as vslow,dump in s6 - T184161 (duration: 00m 58s)

• Marostegui closed this task as Resolved.Mar 12 2018, 7:14 AM
• Marostegui updated the task description. (Show Details)

db1113:3315 and db1113:3316 are now serving traffic.

Content licensed under Creative Commons Attribution-ShareAlike 4.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL