Page MenuHomePhabricator
Create Task
Maniphest T162900

setup naos/WMF6406 as new codfw deployment server
Closed, ResolvedPublic
Actions

Description

This task will track the setup of spare pool system WMF6406 as 'naos' in codfw. The current deployment host mira is out of warranty, and experiencing network card issues (mainboard issues.) WMF6406 was recently used as graphite2003 to copy data, so it will need to have some items updated.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
adds naos to everywhere mira is listedoperations/puppetproduction+21 -1
deployment: sync home dirs from mira to naosoperations/puppetproduction+29 -1
setting naos (new codfw deploy host) ipv6 dnsoperations/dnsmaster+3 -0
setting naos install and site parametersoperations/puppetproduction+7 -2
setting up dns for naosoperations/dnsmaster+4 -4
Customize query in gerrit

Related Objects
Search...

Event Timeline

RobH created this task.Apr 13 2017, 3:36 PM
RobH mentioned this in T162859: Swap NIC on mira.
RobH raised the priority of this task from Medium to High.Apr 13 2017, 3:41 PM

So the switchover from the eqiad deployment host (tin) to the codfw deployment host (mira) was scheduled for approximately April 19th. Ideally naos is online to replace mira before then.

Shifting this to high priority.

RobH updated the task description. (Show Details)Apr 13 2017, 3:45 PM
Papaul reassigned this task from Papaul to RobH.Apr 13 2017, 4:31 PM
Papaul updated the task description. (Show Details)
gerritbot subscribed.Apr 13 2017, 5:02 PM

Change 348103 had a related patch set uploaded (by RobH):
[operations/dns@master] setting up dns for naos

https://gerrit.wikimedia.org/r/348103

gerritbot added a project: Patch-For-Review.Apr 13 2017, 5:02 PM
RobH mentioned this in T161538: Plug in ex-graphite2001 SSDs to recover coal data.Apr 13 2017, 5:04 PM
gerritbot added a comment.Apr 13 2017, 5:05 PM

Change 348103 merged by RobH:
[operations/dns@master] setting up dns for naos

https://gerrit.wikimedia.org/r/348103

gerritbot added a comment.Apr 13 2017, 5:09 PM

Change 348104 had a related patch set uploaded (by RobH):
[operations/puppet@production] setting naos install and site parameters

https://gerrit.wikimedia.org/r/348104

RobH removed a project: Patch-For-Review.Apr 13 2017, 5:10 PM
RobH updated the task description. (Show Details)
gerritbot added a comment.Apr 13 2017, 5:12 PM

Change 348104 merged by RobH:
[operations/puppet@production] setting naos install and site parameters

https://gerrit.wikimedia.org/r/348104

RobH reassigned this task from RobH to Papaul.Apr 13 2017, 5:15 PM
RobH updated the task description. (Show Details)

I just noticed the network port wasn't labeled on the switch with the asset tag, so I need @Papaul to determine what it is:

  • - @Papaul to update this task with the network port this system is plugged into and assign back to @RobH
RobH updated the task description. (Show Details)Apr 13 2017, 5:17 PM
Papaul reassigned this task from Papaul to RobH.Apr 13 2017, 6:04 PM

ge-5/0/15

RobH updated the task description. (Show Details)Apr 13 2017, 6:14 PM
RobH updated the task description. (Show Details)Apr 13 2017, 11:29 PM
gerritbot added a comment.Apr 17 2017, 3:43 PM

Change 348474 had a related patch set uploaded (by RobH):
[operations/dns@master] setting naos (new codfw deploy host) ipv6 dns

https://gerrit.wikimedia.org/r/348474

gerritbot added a project: Patch-For-Review.Apr 17 2017, 3:43 PM
gerritbot added a comment.Apr 17 2017, 3:46 PM

Change 348474 merged by RobH:
[operations/dns@master] setting naos (new codfw deploy host) ipv6 dns

https://gerrit.wikimedia.org/r/348474

gerritbot added a comment.Apr 17 2017, 3:53 PM

Change 348478 had a related patch set uploaded (by RobH):
[operations/puppet@production] adds naos to everywhere mira is listed

https://gerrit.wikimedia.org/r/348478

RobH updated the task description. (Show Details)Apr 17 2017, 3:59 PM
RobH added a comment.Apr 17 2017, 4:01 PM

So my latest patchset https://phabricator.wikimedia.org/T162900 has appending in naos for everything that has mira. I'm not 100% on some of the files, and if simple merging may require further update elsewhere (like touching scap config).

I've added for review some folks on the patchset, and it will be listed on our async meeting notes as a blocker.

gerritbot added a comment.Apr 18 2017, 4:49 PM

Change 348478 merged by Filippo Giunchedi:
[operations/puppet@production] adds naos to everywhere mira is listed

https://gerrit.wikimedia.org/r/348478

fgiunchedi subscribed.Apr 18 2017, 5:49 PM

I've merged @RobH patch and ran puppet on naos, issues I've encountered so far:

The UID issues needs a more permanent fix but I've manually fixed it for now

Dzahn updated the task description. (Show Details)Apr 18 2017, 11:47 PM
gerritbot added a comment.Apr 19 2017, 12:25 AM

Change 348886 had a related patch set uploaded (by Dzahn):
[operations/puppet@production] deployment: sync home dirs from mira to naos

https://gerrit.wikimedia.org/r/348886

Dzahn subscribed.EditedApr 19 2017, 12:51 AM
  • backups: confirmed with bconsole that naos now exists in Bacula with the same backup sets (/home and /srv/deployment are backed up on deployment servers)
  • UID issue: see this https://gerrit.wikimedia.org/r/#/c/348884/
gerritbot added a comment.Apr 19 2017, 1:03 AM

Change 348886 merged by Dzahn:
[operations/puppet@production] deployment: sync home dirs from mira to naos

https://gerrit.wikimedia.org/r/348886

Stashbot subscribed.Apr 19 2017, 1:47 AM

Mentioned in SAL (#wikimedia-operations) [2017-04-19T01:47:27Z] <mutante> rsyncing /home from mira to naos (T162900)

Dzahn updated the task description. (Show Details)Apr 19 2017, 1:59 AM
Dzahn updated the task description. (Show Details)
Dzahn updated the task description. (Show Details)Apr 19 2017, 2:02 AM
Dzahn updated the task description. (Show Details)Apr 19 2017, 2:06 AM
Dzahn removed a project: Patch-For-Review.
Dzahn added a subscriber: • demon.
Stashbot added a comment.Apr 19 2017, 11:23 AM

Mentioned in SAL (#wikimedia-operations) [2017-04-19T11:23:40Z] <godog> add naos to git-deploy term on common-infrastructure4 - T162900

fgiunchedi updated the task description. (Show Details)Apr 19 2017, 11:24 AM
fgiunchedi updated the task description. (Show Details)Apr 19 2017, 1:04 PM
fgiunchedi added a comment.Apr 24 2017, 8:56 AM

Followup for trebuchet/mwdeploy fixed uid/gid: https://phabricator.wikimedia.org/T163667

fgiunchedi added a comment.May 5 2017, 9:00 AM

Anything left to do here? naos is effectively in service and mira should be decom'd (either with its NIC swapped or not in T162859)

Dzahn added a comment.May 5 2017, 2:22 PM

I don't think so. Except mira needs a proper decom task with the checkbox-template for decoms on it.

Dzahn closed this task as Resolved.May 5 2017, 2:44 PM
Dzahn mentioned this in T164588: decom mira.

follow-up task for mira created at T164588

closing as resolved

Dzahn mentioned this in T163667: Fix UIDs for deployment server users.Dec 1 2022, 12:56 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits