Page MenuHomePhabricator

Decommission restbase-test environment
Closed, ResolvedPublic

Description

The restbase-test environment (colloquially referred to as "staging") is deployed on old, obsolete, and mismatched hardware that doesn't come close to reflecting what we run in production. There seems to be consensus that we can carry out staging in deployment-prep, use the restbase-dev environment for performance-critical testing, and decommission the restbase-test machines.

  • - all system services confirmed offline from production use
  • - set all icinga checks to maint mode/disabled while reclaim/decommmission takes place.
  • - remove system from all lvs/pybal active configuration
  • - any service group puppet/hiera/dsh config removed
  • - remove site.pp (replace with role::spare::system if system isn't shut down immediately during this process.)

START NON-INTERRUPPTABLE STEPS

  • - disable puppet on host
  • - remove all remaining puppet references (include role::spare)
  • - power down host
  • - disable switch port
  • - switch port assignment noted on this task (for later removal)
  • - remove production dns entries
  • - puppet node clean, puppet node deactivate

END NON-INTERRUPPTABLE STEPS

  • - system disks wiped (by onsite)
  • - IF DECOM: system unracked and decommissioned (by onsite), update racktables with result
  • - IF DECOM: switch port configration removed from switch once system is unracked.
  • - IF DECOM: mgmt dns entries removed.
  • - IF RECLAIM: system added back to spares tracking (by onsite)

Event Timeline

Eevans triaged this task as Medium priority.Feb 7 2018, 9:04 PM

Confirmed in team meeting today; These machines can be decommissioned at the earliest convenience!

Change 415827 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] Decom restbase-test cluster and role

https://gerrit.wikimedia.org/r/415827

Change 415827 merged by Filippo Giunchedi:
[operations/puppet@production] Decom restbase-test cluster and role

https://gerrit.wikimedia.org/r/415827

All hosts in this task and its subtasks are ready for decom (running as spare systems now)

So I don't see any hosts in this task, only sub-tasks. Is there any reason this task needs to stay open?

Cmjohnson claimed this task.
Cmjohnson subscribed.

This was done awhile ago resolving