Page MenuHomePhabricator

Decommission OCG from production
Closed, ResolvedPublic

Description

We need to decommission OCG from WMF production, as it's being actively replaced by different pieces of software (see parent task). It was deconfigured from production yesterday (cf. T177795), so now the next steps are to actually decom the service. This includes but is not limited to:

  • Decom'the ocg.svc.eqiad.wmnet service IP
  • Remove all references in the Puppet tree
  • Remove any references to OCG metrics in Grafana (e.g. the OCG dashboard)
  • Move servers to the spare pool
  • Wipe the servers and decommission (see subtask)

Details

Related Gerrit Patches:
operations/puppet : productionganglia: remove pdf cluster
operations/puppet : productionocg: remove all references from puppet
operations/puppet : productionocg: remove from conftool
operations/dns : masterRemove references to OCG services

Event Timeline

faidon created this task.Oct 11 2017, 1:04 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 11 2017, 1:04 PM

Change 383578 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] ocg: remove from conftool

https://gerrit.wikimedia.org/r/383578

Change 383580 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] ocg: remove all references from puppet

https://gerrit.wikimedia.org/r/383580

Change 383581 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/dns@master] Remove references to OCG services

https://gerrit.wikimedia.org/r/383581

Mentioned in SAL (#wikimedia-operations) [2017-10-11T14:47:43Z] <_joe_> rolling restart of low-traffic pybals in eqiad for T177931

Mentioned in SAL (#wikimedia-operations) [2017-10-11T14:55:21Z] <_joe_> manually removing IPVS entries for ocg on eqiad LBs, T177931

Change 383581 merged by Giuseppe Lavagetto:
[operations/dns@master] Remove references to OCG services

https://gerrit.wikimedia.org/r/383581

Change 383578 merged by Giuseppe Lavagetto:
[operations/puppet@production] ocg: remove from conftool

https://gerrit.wikimedia.org/r/383578

Mentioned in SAL (#wikimedia-operations) [2017-10-11T15:08:49Z] <_joe_> ocg*: stop ocg; mv /srv/deployment /srv/stale, update-rc.d ocg disable, rm /etc/init/ocg.conf - T177931

Mentioned in SAL (#wikimedia-operations) [2017-10-11T15:21:38Z] <_joe_> stopped redis on ocg* T177931

Joe updated the task description. (Show Details)Oct 11 2017, 3:22 PM

Mentioned in SAL (#wikimedia-releng) [2017-10-11T15:32:20Z] <_joe_> removing deployment-pdf01, T177931

Change 383580 merged by Giuseppe Lavagetto:
[operations/puppet@production] ocg: remove all references from puppet

https://gerrit.wikimedia.org/r/383580

Change 383600 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] ganglia: remove pdf cluster

https://gerrit.wikimedia.org/r/383600

Change 383600 merged by Giuseppe Lavagetto:
[operations/puppet@production] ganglia: remove pdf cluster

https://gerrit.wikimedia.org/r/383600

Joe updated the task description. (Show Details)Oct 11 2017, 3:50 PM
Joe closed this task as Resolved.Dec 20 2017, 6:49 AM

Marked as resolved, I don't have much to do here.