Page MenuHomePhabricator
Create Task
Maniphest T196493

rack/setup/install dns200[12].wikimedia.org
Closed, ResolvedPublic
Actions

Description

This task will track the racking, setup, installation, and implementation of the two new replacement dns recursors for codfw, dns2001.wikimedia.org & dns2002.wikimedia.org. Hostnames were cleared with @BBlack, as we're matching the hostnames in use for dns recursor systems in caching sites. These will replace acamar & achernar

Racking Proposal: These two hosts should rack into different racks and rows than one another, but they do not care about the location of existing dns recursors (since they replace their use.)

dns2001:

  • - receive in system on procurement task T195334
  • - rack system with proposed racking plan (see above) & update racktables (include all system info plus location)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - note network port info on task and assign to @RobH for setup
    • end on-site specific steps
  • - network port setup (description, enable, public vlan)
  • - production public dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - handoff for service implementation

dns2002:

  • - receive in system on procurement task T195334
  • - rack system with proposed racking plan (see above) & update racktables (include all system info plus location)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - note network port info on task and assign to @RobH for setup
    • end on-site specific steps
  • - network port setup (description, enable, public vlan)
  • - production public dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - handoff for service implementation

Details

Show related patches Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
Unknown Object (Task)
Unknown Object (Task)
 ResolvedVgutierrezT196493 rack/setup/install dns200[12].wikimedia.org
 ResolvedayounsiT197697 switch port configuration for dns200[1-2]

Event Timeline

RobH triaged this task as Medium priority.Jun 5 2018, 5:59 PM
RobH created this task.
Restricted Application added a project: Traffic. · View Herald TranscriptJun 5 2018, 5:59 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Papaul updated the task description. (Show Details)Jun 7 2018, 8:01 PM
Papaul updated the task description. (Show Details)Jun 7 2018, 8:06 PM
Vgutierrez subscribed.Jun 8 2018, 7:15 AM
BBlack moved this task from Backlog to Hardware on the Traffic board.Jun 11 2018, 5:35 PM
gerritbot subscribed.Jun 12 2018, 4:51 PM

Change 439973 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/dns@master] DNS: Add mgmt DNS entries for dns200[1-2]

https://gerrit.wikimedia.org/r/439973

gerritbot added a project: Patch-For-Review.Jun 12 2018, 4:51 PM
Papaul updated the task description. (Show Details)Jun 12 2018, 4:58 PM
gerritbot added a comment.Jun 14 2018, 6:16 PM

Change 439973 merged by Dzahn:
[operations/dns@master] DNS: Add mgmt DNS entries for dns200[1-2]

https://gerrit.wikimedia.org/r/439973

gerritbot added a comment.Jun 19 2018, 3:08 PM

Change 441055 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/dns@master] DNS: Add production DNS entries for dns200[1-2]

https://gerrit.wikimedia.org/r/441055

Papaul updated the task description. (Show Details)Jun 19 2018, 3:10 PM
Papaul updated the task description. (Show Details)Jun 19 2018, 3:14 PM
gerritbot added a comment.Jun 19 2018, 3:26 PM

Change 441059 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/puppet@production] DHCP: Add MAC address entries for dns200[1-2]

https://gerrit.wikimedia.org/r/441059

gerritbot added a comment.Jun 19 2018, 4:06 PM

Change 441062 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/dns@master] DNS: Fix mgmt asset tag for dns2002

https://gerrit.wikimedia.org/r/441062

gerritbot added a comment.Jun 19 2018, 4:14 PM

Change 441063 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/dns@master] DNS: Add DNS asset tag mgmt for spare servers

https://gerrit.wikimedia.org/r/441063

gerritbot added a comment.Jun 20 2018, 9:35 AM

Change 441177 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] site: add dns2001/dns2002 with role(spare)

https://gerrit.wikimedia.org/r/441177

gerritbot added a comment.Jun 20 2018, 11:50 AM

Change 441177 merged by Dzahn:
[operations/puppet@production] site: add dns2001/dns2002 with role(spare)

https://gerrit.wikimedia.org/r/441177

gerritbot added a comment.Jun 20 2018, 11:55 AM

Change 441055 merged by Dzahn:
[operations/dns@master] DNS: Add production DNS entries for dns200[1-2]

https://gerrit.wikimedia.org/r/441055

gerritbot added a comment.Jun 20 2018, 11:59 AM

Change 441059 merged by Dzahn:
[operations/puppet@production] DHCP: Add MAC address entries for dns200[1-2]

https://gerrit.wikimedia.org/r/441059

gerritbot added a comment.Jun 20 2018, 12:05 PM

Change 441062 merged by Dzahn:
[operations/dns@master] DNS: Fix mgmt asset tag for dns2002

https://gerrit.wikimedia.org/r/441062

ayounsi closed subtask T197697: switch port configuration for dns200[1-2] as Resolved.Jun 25 2018, 7:20 AM
ops-monitoring-bot subscribed.Jun 25 2018, 10:32 AM

Script wmf-auto-reimage was launched by vgutierrez on neodymium.eqiad.wmnet for hosts:

dns2001.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201806251032_vgutierrez_29708_dns2001_wikimedia_org.log.

ops-monitoring-bot added a comment.Jun 25 2018, 10:45 AM

Completed auto-reimage of hosts:

['dns2001.wikimedia.org']

Of which those FAILED:

['dns2001.wikimedia.org']
gerritbot added a comment.Jun 25 2018, 10:48 AM

Change 441822 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] install_server: Set netboot config for dns200[12].wikimedia.org

https://gerrit.wikimedia.org/r/441822

gerritbot added a comment.Jun 25 2018, 10:54 AM

Change 441822 merged by Vgutierrez:
[operations/puppet@production] install_server: Set netboot config for dns200[12].wikimedia.org

https://gerrit.wikimedia.org/r/441822

ops-monitoring-bot added a comment.Jun 25 2018, 10:59 AM

Script wmf-auto-reimage was launched by vgutierrez on neodymium.eqiad.wmnet for hosts:

dns2001.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201806251059_vgutierrez_1222_dns2001_wikimedia_org.log.

ops-monitoring-bot added a comment.Jun 25 2018, 11:20 AM

Completed auto-reimage of hosts:

['dns2001.wikimedia.org']

and were ALL successful.

ops-monitoring-bot added a comment.Jun 25 2018, 11:23 AM

Script wmf-auto-reimage was launched by vgutierrez on neodymium.eqiad.wmnet for hosts:

dns2002.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201806251122_vgutierrez_7465_dns2002_wikimedia_org.log.

Vgutierrez updated the task description. (Show Details)Jun 25 2018, 11:23 AM
Vgutierrez updated the task description. (Show Details)Jun 25 2018, 11:43 AM
ops-monitoring-bot added a comment.Jun 25 2018, 11:44 AM

Completed auto-reimage of hosts:

['dns2002.wikimedia.org']

and were ALL successful.

gerritbot added a comment.Jun 25 2018, 11:47 AM

Change 441834 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] site: set proper role for dns200[12].wikimedia.org

https://gerrit.wikimedia.org/r/441834

gerritbot added a comment.Jun 25 2018, 12:33 PM

Change 441834 merged by Vgutierrez:
[operations/puppet@production] site: set proper role for dns200[12].wikimedia.org

https://gerrit.wikimedia.org/r/441834

ops-monitoring-bot added a comment.Jun 25 2018, 12:41 PM

Script wmf-auto-reimage was launched by vgutierrez on neodymium.eqiad.wmnet for hosts:

dns2001.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201806251241_vgutierrez_23783_dns2001_wikimedia_org.log.

ops-monitoring-bot added a comment.Jun 25 2018, 12:53 PM

Completed auto-reimage of hosts:

['dns2001.wikimedia.org']

Of which those FAILED:

['dns2001.wikimedia.org']
gerritbot added a comment.Jun 25 2018, 1:14 PM

Change 441860 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] standard: add dns200[12] to codfw ntp peer list

https://gerrit.wikimedia.org/r/441860

Papaul reassigned this task from Papaul to Vgutierrez.Jun 25 2018, 2:36 PM
Papaul subscribed.
Papaul unsubscribed.
gerritbot added a comment.Jun 25 2018, 2:52 PM

Change 441860 merged by Vgutierrez:
[operations/puppet@production] standard: add dns200[12] to codfw ntp peer list

https://gerrit.wikimedia.org/r/441860

gerritbot added a comment.Jun 25 2018, 3:26 PM

Change 441891 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] replace achernar with dns2002

https://gerrit.wikimedia.org/r/441891

gerritbot added a comment.Jun 25 2018, 3:50 PM

Change 441891 merged by Vgutierrez:
[operations/puppet@production] conftool-data: Add dns200[12] to pdns_recursor service

https://gerrit.wikimedia.org/r/441891

RobH removed a project: ops-codfw.Jun 25 2018, 4:18 PM
gerritbot added a comment.Jun 25 2018, 4:37 PM

Change 441911 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] conftool-data: Remove achernar and acamar from pdns_recursor

https://gerrit.wikimedia.org/r/441911

gerritbot added a comment.Jun 25 2018, 4:40 PM

Change 441911 merged by Vgutierrez:
[operations/puppet@production] conftool-data: Remove achernar and acamar from pdns_recursor

https://gerrit.wikimedia.org/r/441911

gerritbot added a comment.Jun 25 2018, 4:45 PM

Change 441912 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] standard: Remove acamar and achernar from ntp peer list

https://gerrit.wikimedia.org/r/441912

gerritbot added a comment.Jun 25 2018, 4:48 PM

Change 441912 merged by Vgutierrez:
[operations/puppet@production] standard: Remove acamar and achernar from ntp peer list

https://gerrit.wikimedia.org/r/441912

gerritbot added a comment.Jun 25 2018, 5:00 PM

Change 441916 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] lvs: use the new dns200[12] recursive DNS servers

https://gerrit.wikimedia.org/r/441916

gerritbot added a comment.Jun 25 2018, 5:03 PM

Change 441918 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] hieradata: Get rid of acamar and achenar references

https://gerrit.wikimedia.org/r/441918

gerritbot added a comment.Jun 25 2018, 5:07 PM

Change 441919 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] smokeping: Replace acamar & achernar with dns2001 and dns2002

https://gerrit.wikimedia.org/r/441919

gerritbot added a comment.Jun 26 2018, 7:20 AM

Change 441916 merged by Vgutierrez:
[operations/puppet@production] lvs: use the new dns200[12] recursive DNS servers

https://gerrit.wikimedia.org/r/441916

gerritbot added a comment.Jun 26 2018, 7:49 AM

Change 441919 merged by Vgutierrez:
[operations/puppet@production] smokeping: Replace acamar & achernar with dns2001 and dns2002

https://gerrit.wikimedia.org/r/441919

gerritbot added a comment.Jun 26 2018, 7:53 AM

Change 441918 merged by Vgutierrez:
[operations/puppet@production] hieradata: Get rid of acamar and achernar references

https://gerrit.wikimedia.org/r/441918

Stashbot subscribed.Jun 26 2018, 7:58 AM

Mentioned in SAL (#wikimedia-operations) [2018-06-26T07:58:26Z] <vgutierrez> Replaced acamar and achernar with dns200[12] as main lvs name servers in codfw - T196493

Vgutierrez closed this task as Resolved.Jun 27 2018, 8:05 AM
Vgutierrez updated the task description. (Show Details)
Vgutierrez mentioned this in T198286: Decommission acamar and achernar.
Vgutierrez removed a project: Patch-For-Review.Jun 27 2018, 8:23 AM
• Vvjjkkii reopened subtask T197697: switch port configuration for dns200[1-2] as Open.Jul 1 2018, 1:02 AM
• Vvjjkkii renamed this task from rack/setup/install dns200[12].wikimedia.org to 6kbaaaaaaa.Jul 1 2018, 1:05 AM
• Vvjjkkii reopened this task as Open.
• Vvjjkkii removed Vgutierrez as the assignee of this task.
• Vvjjkkii raised the priority of this task from Medium to High.
• Vvjjkkii added projects: CheckUser, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), Tamil-Sites, Gamepress, Hashtags, Jade, KartoEditor, Language-2018-Apr-June, New-Editor-Experiences, Mail, TCB-Team (now WMDE-TechWish).
• Vvjjkkii updated the task description. (Show Details)
• Vvjjkkii removed subscribers: gerritbot, Aklapper.
CommunityTechBot renamed this task from 6kbaaaaaaa to rack/setup/install dns200[12].wikimedia.org.Jul 2 2018, 6:48 AM
CommunityTechBot closed this task as Resolved.
CommunityTechBot assigned this task to Vgutierrez.
CommunityTechBot lowered the priority of this task from High to Medium.
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot removed projects: TCB-Team (now WMDE-TechWish), Mail, New-Editor-Experiences, Language-2018-Apr-June, KartoEditor, Jade, Hashtags, Gamepress, Tamil-Sites, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), CheckUser.
CommunityTechBot added subscribers: gerritbot, Aklapper.
• ema closed subtask T197697: switch port configuration for dns200[1-2] as Resolved.Jul 2 2018, 9:33 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL