Page MenuHomePhabricator
Create Task
Maniphest T196376

Productionize old/temporary eqiad sanitariums
Closed, ResolvedPublic
Actions

Description

The following hosts should be productionized once they stop being used

They are old/temporary sanitarium hosts.

  • db1102 (D8) (backup source host T201392)
  • db1120 (C5) pooled in x1

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

• Marostegui changed the task status from Open to Stalled.Jun 4 2018, 3:13 PM
• Marostegui triaged this task as Medium priority.
• Marostegui created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 4 2018, 3:13 PM
• Marostegui moved this task from Triage to Pending comment on the DBA board.Jun 4 2018, 3:13 PM
• Marostegui mentioned this in T190704: Convert all sanitarium hosts to multi-instance and increase its reliability/redundancy.
gerritbot subscribed.Jun 5 2018, 9:08 AM

Change 437418 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Set db1120 to spare

https://gerrit.wikimedia.org/r/437418

gerritbot added a project: Patch-For-Review.Jun 5 2018, 9:08 AM
gerritbot added a comment.Jun 5 2018, 9:17 AM

Change 437418 merged by Marostegui:
[operations/puppet@production] mariadb: Set db1120 to spare

https://gerrit.wikimedia.org/r/437418

ops-monitoring-bot subscribed.Jun 5 2018, 9:17 AM

Script wmf-auto-reimage was launched by marostegui on neodymium.eqiad.wmnet for hosts:

['db1120.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201806050917_marostegui_5963.log.

gerritbot added a comment.Jun 5 2018, 9:21 AM

Change 437423 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/software@master] sX.hosts: Remove db1120

https://gerrit.wikimedia.org/r/437423

gerritbot added a comment.Jun 5 2018, 9:31 AM

Change 437423 merged by jenkins-bot:
[operations/software@master] sX.hosts: Remove db1120

https://gerrit.wikimedia.org/r/437423

ops-monitoring-bot added a comment.Jun 5 2018, 9:34 AM

Completed auto-reimage of hosts:

['db1120.eqiad.wmnet']

and were ALL successful.

• Marostegui changed the task status from Stalled to Open.Jun 5 2018, 9:35 AM
• Marostegui updated the task description. (Show Details)
• Marostegui mentioned this in T196527: Clean up sanitarium_multisource related code.Jun 6 2018, 8:46 AM
gerritbot added a comment.Jun 6 2018, 8:58 AM

Change 437687 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Convert db1116 to spare

https://gerrit.wikimedia.org/r/437687

gerritbot added a comment.Jun 6 2018, 9:46 AM

Change 437687 merged by Marostegui:
[operations/puppet@production] mariadb: Convert db1116 to spare

https://gerrit.wikimedia.org/r/437687

ops-monitoring-bot added a comment.Jun 6 2018, 9:47 AM

Script wmf-auto-reimage was launched by marostegui on neodymium.eqiad.wmnet for hosts:

['db1116.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201806060947_marostegui_19240.log.

ops-monitoring-bot added a comment.Jun 6 2018, 10:05 AM

Completed auto-reimage of hosts:

['db1116.eqiad.wmnet']

and were ALL successful.

• Marostegui updated the task description. (Show Details)Jun 6 2018, 10:06 AM
gerritbot added a comment.Jun 6 2018, 10:08 AM

Change 437711 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/software@master] sX.hosts: Remove db1116

https://gerrit.wikimedia.org/r/437711

gerritbot added a comment.Jun 6 2018, 10:10 AM

Change 437711 merged by jenkins-bot:
[operations/software@master] sX.hosts: Remove db1116

https://gerrit.wikimedia.org/r/437711

gerritbot added a comment.Jun 6 2018, 10:44 AM

Change 437720 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Set db1095 as spare, remove unused code

https://gerrit.wikimedia.org/r/437720

gerritbot added a comment.Jun 26 2018, 5:17 AM

Change 437720 merged by Marostegui:
[operations/puppet@production] mariadb: Set db1095 as spare, remove unused code

https://gerrit.wikimedia.org/r/437720

ops-monitoring-bot added a comment.Jun 26 2018, 5:22 AM

Script wmf-auto-reimage was launched by marostegui on neodymium.eqiad.wmnet for hosts:

['db1095.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201806260522_marostegui_574.log.

gerritbot added a comment.Jun 26 2018, 5:33 AM

Change 442013 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/software@master] s*.hosts: Remove db1095

https://gerrit.wikimedia.org/r/442013

gerritbot added a comment.Jun 26 2018, 5:35 AM

Change 442013 merged by jenkins-bot:
[operations/software@master] s*.hosts: Remove db1095

https://gerrit.wikimedia.org/r/442013

ops-monitoring-bot added a comment.Jun 26 2018, 5:43 AM

Completed auto-reimage of hosts:

['db1095.eqiad.wmnet']

and were ALL successful.

• Marostegui updated the task description. (Show Details)Jun 26 2018, 5:43 AM
gerritbot added a comment.Jun 26 2018, 6:18 AM

Change 442016 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Set db1102 as spare

https://gerrit.wikimedia.org/r/442016

gerritbot added a comment.Jun 26 2018, 6:20 AM

Change 442017 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/software@master] s*.hosts: Remove db1102

https://gerrit.wikimedia.org/r/442017

gerritbot added a comment.Jun 26 2018, 6:22 AM

Change 442017 merged by jenkins-bot:
[operations/software@master] s*.hosts: Remove db1102

https://gerrit.wikimedia.org/r/442017

gerritbot added a comment.Jun 26 2018, 6:23 AM

Change 442016 merged by Marostegui:
[operations/puppet@production] mariadb: Set db1102 as spare

https://gerrit.wikimedia.org/r/442016

ops-monitoring-bot added a comment.Jun 26 2018, 6:32 AM

Script wmf-auto-reimage was launched by marostegui on neodymium.eqiad.wmnet for hosts:

['db1102.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201806260632_marostegui_16518.log.

ops-monitoring-bot added a comment.Jun 26 2018, 6:49 AM

Completed auto-reimage of hosts:

['db1102.eqiad.wmnet']

and were ALL successful.

• Marostegui updated the task description. (Show Details)Jun 26 2018, 7:37 AM
• Vvjjkkii renamed this task from Productionize old/temporary eqiad sanitariums to fobaaaaaaa.Jul 1 2018, 1:06 AM
• Vvjjkkii raised the priority of this task from Medium to High.
• Vvjjkkii added projects: CheckUser, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), Tamil-Sites, Gamepress, Hashtags, Jade, KartoEditor, Language-2018-Apr-June, New-Editor-Experiences, Mail, TCB-Team (now WMDE-TechWish).
• Vvjjkkii updated the task description. (Show Details)
• Vvjjkkii removed subscribers: gerritbot, Aklapper.
• Marostegui renamed this task from fobaaaaaaa to Productionize old/temporary eqiad sanitariums.Jul 2 2018, 5:02 AM
• Marostegui lowered the priority of this task from High to Medium.
• Marostegui removed projects: TCB-Team (now WMDE-TechWish), Mail, New-Editor-Experiences, Language-2018-Apr-June, KartoEditor, Jade, Hashtags, Gamepress, Tamil-Sites, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), CheckUser.
• Marostegui updated the task description. (Show Details)
• Marostegui mentioned this in T187850: Cleanup remaining WikipediaMobileFirefoxOS references.Jul 6 2018, 4:51 AM
• Marostegui mentioned this in T199224: Test database master switchover script on codfw.Jul 10 2018, 1:38 PM
• Marostegui added a comment.Jul 20 2018, 5:46 AM

I will take db1120 (row C) instead of db1116 as db1116 is in row A (like db1069 - x1 master).
The slave db1064 is in row D, so that way we have all the 3 hosts in different rows.

gerritbot added a comment.Jul 20 2018, 5:48 AM

Change 447021 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] site.pp: Specify db1120 future location

https://gerrit.wikimedia.org/r/447021

gerritbot added a comment.Jul 20 2018, 5:49 AM

Change 447021 merged by Marostegui:
[operations/puppet@production] site.pp: Specify db1120 future location

https://gerrit.wikimedia.org/r/447021

gerritbot added a comment.Jul 25 2018, 2:29 PM

Change 447811 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] mariadb: Add db1120 to x1

https://gerrit.wikimedia.org/r/447811

gerritbot added a comment.Jul 25 2018, 2:33 PM

Change 447811 merged by Marostegui:
[operations/puppet@production] mariadb: Add db1120 to x1

https://gerrit.wikimedia.org/r/447811

ops-monitoring-bot added a comment.Jul 25 2018, 2:36 PM

Script wmf-auto-reimage was launched by marostegui on neodymium.eqiad.wmnet for hosts:

['db1120.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201807251436_marostegui_14986.log.

ops-monitoring-bot added a comment.Jul 25 2018, 2:54 PM

Completed auto-reimage of hosts:

['db1120.eqiad.wmnet']

and were ALL successful.

gerritbot added a comment.Jul 30 2018, 6:31 AM

Change 449118 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] db1120: Enable notifications

https://gerrit.wikimedia.org/r/449118

gerritbot added a comment.Jul 30 2018, 6:33 AM

Change 449118 merged by Marostegui:
[operations/puppet@production] db1120: Enable notifications

https://gerrit.wikimedia.org/r/449118

gerritbot added a comment.Jul 30 2018, 6:39 AM

Change 449123 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Add db1120 to x1

https://gerrit.wikimedia.org/r/449123

gerritbot added a comment.Jul 30 2018, 7:26 AM

Change 449123 merged by jenkins-bot:
[operations/mediawiki-config@master] db-eqiad,db-codfw.php: Add db1120 to x1

https://gerrit.wikimedia.org/r/449123

Stashbot subscribed.Jul 30 2018, 7:28 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-30T07:28:13Z] <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Add db1120 to x1 T196376 (duration: 00m 59s)

Stashbot added a comment.Jul 30 2018, 7:29 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-30T07:29:21Z] <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Add db1120 to x1 T196376 (duration: 00m 54s)

• Marostegui updated the task description. (Show Details)Jul 30 2018, 7:54 AM

db1120 is now serving traffic in x1

• Marostegui updated the task description. (Show Details)Aug 7 2018, 9:19 AM
jcrespo updated the task description. (Show Details)Aug 7 2018, 9:58 AM
gerritbot added a comment.Aug 7 2018, 2:07 PM

Change 450990 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/mediawiki-config@master] mariadb: Repool es1019 and db1102 with low load after maintenance

https://gerrit.wikimedia.org/r/450990

gerritbot added a comment.Aug 7 2018, 2:15 PM

Change 450990 merged by jenkins-bot:
[operations/mediawiki-config@master] mariadb: Repool es1019 and db1102 with low load after maintenance

https://gerrit.wikimedia.org/r/450990

gerritbot added a comment.Aug 7 2018, 2:19 PM

Change 450994 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/mediawiki-config@master] mariadb: Repool db1081 with low load after maintenance

https://gerrit.wikimedia.org/r/450994

gerritbot added a comment.Aug 7 2018, 2:23 PM

Change 450994 merged by jenkins-bot:
[operations/mediawiki-config@master] mariadb: Repool db1081 with low load after maintenance

https://gerrit.wikimedia.org/r/450994

• Marostegui moved this task from Pending comment to Backlog on the DBA board.Sep 12 2018, 5:23 AM
jcrespo claimed this task.Sep 27 2018, 1:59 PM
jcrespo moved this task from Backlog to In progress on the DBA board.

I am going to take the spare host and use it to finish eqiad backups (temporary) setup.

gerritbot added a comment.Sep 27 2018, 2:06 PM

Change 463268 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mariadb backups: Convert db1116 into an eqiad backup source host

https://gerrit.wikimedia.org/r/463268

gerritbot added a comment.Sep 28 2018, 8:10 AM

Change 463268 merged by Jcrespo:
[operations/puppet@production] mariadb backups: Convert db1116 into an eqiad backup source host

https://gerrit.wikimedia.org/r/463268

jcrespo closed this task as Resolved.Sep 28 2018, 2:42 PM
jcrespo reassigned this task from jcrespo to • Marostegui.

All hosts setup and in production usage.

jcrespo updated the task description. (Show Details)Sep 28 2018, 2:42 PM
• Marostegui updated the task description. (Show Details)Sep 28 2018, 2:43 PM
• Marostegui updated the task description. (Show Details)
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL