Page MenuHomePhabricator

Migrate dns caches to stretch
Closed, ResolvedPublic

Description

Various stretch-specific puppetization and operational issues for role::recursor, which encapsulates recursive DNS + NTP server functionalities have been resolved during the process of installing the new recdns boxes in eqsin and ulsfo (dns[45]00[12]). The new configuration is considerably better and it's time to upgrade the others as we're able

  • eqiad - chromium
  • eqiad - hydrogen
  • codfw - acamar
  • codfw - achernar
  • esams - maerlant
  • esams - nescio

Some of these are also due for hardware refresh soon, so in some cases it may make more sense to do this as part of that refresh (with the new replacement hosts taking on the standard naming scheme dnsS00N).

Details

Related Gerrit Patches:

Event Timeline

BBlack triaged this task as Medium priority.Feb 12 2018, 4:26 PM
BBlack created this task.
Restricted Application added a project: Operations. · View Herald TranscriptFeb 12 2018, 4:26 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
BBlack moved this task from Triage to DNS Infra on the Traffic board.Feb 13 2018, 1:20 PM
ema added a subscriber: ema.Apr 13 2018, 7:56 AM

Script wmf-auto-reimage was launched by vgutierrez on neodymium.eqiad.wmnet for hosts:

nescio.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201804130858_vgutierrez_27111_nescio_wikimedia_org.log.

Completed auto-reimage of hosts:

['nescio.wikimedia.org']

Of which those FAILED:

['nescio.wikimedia.org']
Vgutierrez updated the task description. (Show Details)Apr 13 2018, 10:19 AM
Vgutierrez updated the task description. (Show Details)

Change 425994 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] install_server: Reimage maerlant.wikimedia.org as stretch

https://gerrit.wikimedia.org/r/425994

Change 425994 merged by Vgutierrez:
[operations/puppet@production] install_server: Reimage maerlant.wikimedia.org as stretch

https://gerrit.wikimedia.org/r/425994

Script wmf-auto-reimage was launched by vgutierrez on neodymium.eqiad.wmnet for hosts:

maerlant.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201804131043_vgutierrez_18025_maerlant_wikimedia_org.log.

Completed auto-reimage of hosts:

['maerlant.wikimedia.org']

Of which those FAILED:

['maerlant.wikimedia.org']
Vgutierrez updated the task description. (Show Details)Apr 13 2018, 12:29 PM

Change 426851 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] install_server: Reimage achernar as stretch

https://gerrit.wikimedia.org/r/426851

Change 426851 merged by Vgutierrez:
[operations/puppet@production] install_server: Reimage achernar as stretch

https://gerrit.wikimedia.org/r/426851

Mentioned in SAL (#wikimedia-operations) [2018-04-16T07:39:23Z] <vgutierrez> Depool and reimage achernar.wikimedia.org - T187090

Script wmf-auto-reimage was launched by vgutierrez on neodymium.eqiad.wmnet for hosts:

achernar.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201804160746_vgutierrez_21034_achernar_wikimedia_org.log.

Completed auto-reimage of hosts:

['achernar.wikimedia.org']

Of which those FAILED:

['achernar.wikimedia.org']
Vgutierrez updated the task description. (Show Details)Apr 16 2018, 9:26 AM

Change 426876 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] install_server: Reimage acamar as stretch

https://gerrit.wikimedia.org/r/426876

Change 426876 merged by Vgutierrez:
[operations/puppet@production] install_server: Reimage acamar as stretch

https://gerrit.wikimedia.org/r/426876

Mentioned in SAL (#wikimedia-operations) [2018-04-16T09:49:54Z] <vgutierrez> Depool and reimage acamar as stretch - T187090

Script wmf-auto-reimage was launched by vgutierrez on neodymium.eqiad.wmnet for hosts:

acamar.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201804160957_vgutierrez_17117_acamar_wikimedia_org.log.

Completed auto-reimage of hosts:

['acamar.wikimedia.org']

Of which those FAILED:

['acamar.wikimedia.org']
Vgutierrez updated the task description. (Show Details)Apr 16 2018, 11:55 AM

Change 426894 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] install_server: Reimage hydrogen as stretch

https://gerrit.wikimedia.org/r/426894

Change 426895 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] Remove hydrogen from eqiad LVS name server config

https://gerrit.wikimedia.org/r/426895

Change 426894 merged by Vgutierrez:
[operations/puppet@production] install_server: Reimage hydrogen as stretch

https://gerrit.wikimedia.org/r/426894

Change 426895 merged by Vgutierrez:
[operations/puppet@production] Remove hydrogen from eqiad LVS name server config

https://gerrit.wikimedia.org/r/426895

Mentioned in SAL (#wikimedia-operations) [2018-04-16T12:11:39Z] <vgutierrez> Depool and reimage hydrogen as stretch - T187090

Script wmf-auto-reimage was launched by vgutierrez on neodymium.eqiad.wmnet for hosts:

hydrogen.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201804161217_vgutierrez_15753_hydrogen_wikimedia_org.log.

Completed auto-reimage of hosts:

['hydrogen.wikimedia.org']

Of which those FAILED:

['hydrogen.wikimedia.org']

Change 426940 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] Revert "Remove hydrogen from eqiad LVS name server config"

https://gerrit.wikimedia.org/r/426940

Change 426940 merged by Vgutierrez:
[operations/puppet@production] Revert "Remove hydrogen from eqiad LVS name server config"

https://gerrit.wikimedia.org/r/426940

Vgutierrez updated the task description. (Show Details)Apr 16 2018, 3:04 PM

Change 427061 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] install_server: Reimage chromium as stretch

https://gerrit.wikimedia.org/r/427061

Change 427062 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] Remove chromium from eqiad LVS name server config

https://gerrit.wikimedia.org/r/427062

Change 427061 merged by Vgutierrez:
[operations/puppet@production] install_server: Reimage chromium as stretch

https://gerrit.wikimedia.org/r/427061

Change 427062 merged by Vgutierrez:
[operations/puppet@production] Remove chromium from eqiad LVS name server config

https://gerrit.wikimedia.org/r/427062

Mentioned in SAL (#wikimedia-operations) [2018-04-17T06:47:45Z] <vgutierrez> Depool and reimage chromium as stretch - T187090

Script wmf-auto-reimage was launched by vgutierrez on neodymium.eqiad.wmnet for hosts:

chromium.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201804170653_vgutierrez_26909_chromium_wikimedia_org.log.

Completed auto-reimage of hosts:

['chromium.wikimedia.org']

Of which those FAILED:

['chromium.wikimedia.org']

Change 427076 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] Revert "Remove chromium from eqiad LVS name server config" chromium has been reimaged successfully.

https://gerrit.wikimedia.org/r/427076

Change 427076 merged by Vgutierrez:
[operations/puppet@production] Revert "Remove chromium from eqiad LVS name server config"

https://gerrit.wikimedia.org/r/427076

Vgutierrez closed this task as Resolved.Apr 17 2018, 8:51 AM
Vgutierrez claimed this task.
Vgutierrez updated the task description. (Show Details)

Change 427101 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] ntp: Cleanup jessie only code

https://gerrit.wikimedia.org/r/427101

Change 427101 merged by BBlack:
[operations/puppet@production] ntp: Cleanup jessie only code

https://gerrit.wikimedia.org/r/427101