decommission wdqs100[12]
Open, LowPublic

Description

Once wdqs100[45] are setup and serving traffic (T171210), wdqs100[12] can be decommissioned (see docs).

  • - all system services confirmed offline from production use
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/hiera/dsh config removed
  • - remove site.pp (replace with role::spare if system isn't shut down immediately during this process.) -> replaced with role::spare::system

START NON-INTERRUPPTABLE STEPS

  • - disable puppet on hosts
  • - remove all remaining puppet references (include role::spare)
  • - power down hosts
  • - disable switch ports
  • - switch port assignment noted on this task (for later removal) wdqs1001:asw2-d-eqiad:ge-3/0/15 & wdqs1002:asw-c-eqiad:ge-7/0/12
  • - remove production dns entries
  • - puppet node clean, puppet node deactivate, salt key removed

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - system unracked and decommissioned (by onsite), update racktables with result
  • - switch port configration removed from switch once system is unracked.
  • - mgmt dns entries removed.
Gehel created this task.Sep 11 2017, 4:12 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 11 2017, 4:12 PM
Gehel updated the task description. (Show Details)Sep 11 2017, 4:13 PM
Gehel updated the task description. (Show Details)

Mentioned in SAL (#wikimedia-operations) [2017-09-13T16:28:31Z] <gehel> starting decommissioning of wdqs100[12] - T175595

Change 377797 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] wdqs - decommissioning wdqs100[12]

https://gerrit.wikimedia.org/r/377797

Was ldf server moved from wdqs1001? If not we should move it first thing.

Gehel updated the task description. (Show Details)Sep 13 2017, 4:36 PM

Change 377797 merged by Gehel:
[operations/puppet@production] wdqs - decommissioning wdqs100[12]

https://gerrit.wikimedia.org/r/377797

Gehel reassigned this task from Gehel to RobH.
Gehel added a subscriber: RobH.

@RobH I think my job is done here, let me know if you need anything else from me.

Gehel updated the task description. (Show Details)Sep 13 2017, 4:56 PM

Change 377801 had a related patch set uploaded (by Gehel; owner: Gehel):
[wikidata/query/deploy@master] decomission wdqs100[12]

https://gerrit.wikimedia.org/r/377801

Change 377801 merged by Smalyshev:
[wikidata/query/deploy@master] decomission wdqs100[12]

https://gerrit.wikimedia.org/r/377801

RobH added a subscriber: Cmjohnson.Sep 20 2017, 5:59 PM

I cannot find wdqs1001 on the network switch stack for row d (either old asw or new asw2 stack). @Cmjohnson will need to unplug the actual network cable.

Change 379282 had a related patch set uploaded (by RobH; owner: RobH):
[operations/dns@master] decom wdqs100[12]

https://gerrit.wikimedia.org/r/379282

RobH updated the task description. (Show Details)Sep 20 2017, 6:02 PM

@RobH updated ge-3/0/15 still had asset tag listed.

Change 379282 merged by RobH:
[operations/dns@master] decom wdqs100[12]

https://gerrit.wikimedia.org/r/379282

Change 379287 had a related patch set uploaded (by RobH; owner: RobH):
[operations/puppet@production] decom of wdqs100[12]

https://gerrit.wikimedia.org/r/379287

Change 379287 merged by RobH:
[operations/puppet@production] decom of wdqs100[12]

https://gerrit.wikimedia.org/r/379287

RobH reassigned this task from RobH to Cmjohnson.EditedSep 20 2017, 6:10 PM

Assigned to @Cmjohnson for onsite followup.

RobH edited projects, added ops-eqiad; removed Patch-For-Review.
RobH moved this task from Backlog to Decommission on the ops-eqiad board.
RobH updated the task description. (Show Details)Sep 20 2017, 6:15 PM
RobH triaged this task as Low priority.

Removing this from the WDQS board, nothing more to do on our side...