Page MenuHomePhabricator

Deprovision Diamond collectors no longer in use
Closed, ResolvedPublic

Description

With T177196: Port non-deprecated Diamond collectors to Prometheus we've deprecated several Diamond collectors now, we can thus proceed to deprovision / decom the ones that are no longer in use. Time wise this should happen in 10-12 weeks to give Prometheus enough time to collect data.

For hosts running only basic collectors (e.g. system stats already replaced by node-exporter) we should consider removing Diamond altogether.

Collectors to deprecate:

  • memcached modules/memcached/manifests/init.pp
  • nginx: modules/debug_proxy/manifests/init.pp modules/dynamicproxy/manifests/init.pp modules/profile/manifests/docker/registry.pp modules/puppetmaster/manifests/puppetdb.pp
  • ircdstats: modules/mw_rc_irc/manifests/ircserver.pp
  • nutcracker: modules/nutcracker/manifests/monitoring.pp
  • apachestatussimple: modules/phabricator/manifests/monitoring.pp
  • etherpad: modules/profile/manifests/etherpad.pp
  • kubernetes: modules/profile/manifests/kubernetes/master.pp
  • redis: modules/profile/manifests/redis/master.pp modules/profile/manifests/redis/multidc.ppmodules/profile/manifests/redis/slave.pp
  • udpcollector: modules/profile/manifests/statsd.pp
  • pybal: modules/pybal/manifests/monitoring.pp
  • rabbitmq: modules/rabbitmq/manifests/monitor.pp
  • openldap/corp: modules/role/manifests/openldap/corp.pp
  • openldap/labs:modules/role/manifests/openldap/labs.pp
  • ntpd: modules/ntp/manifests/daemon.pp
  • nfsdcollector: modules/labstore/manifests/monitoring/nfsd.pp
  • DirectorySize: modules/role/manifests/labs/nfs/secondary.pp

Diamond in Toolforge is tracked via T210991

Diamond in Cloud VPS is tracked via T210993

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 474138 merged by Filippo Giunchedi:
[operations/puppet@production] grafana: deprecate Diamond metrics in swift dashboard

https://gerrit.wikimedia.org/r/474138

Change 474280 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove Diamond from Swift backends

https://gerrit.wikimedia.org/r/474280

Change 473302 merged by Cwhite:
[operations/puppet@production] diamond: remove diamond::collector::nginx

https://gerrit.wikimedia.org/r/473302

Change 474309 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet/nginx@master] nginx: remove diamond::collector::nginx reference

https://gerrit.wikimedia.org/r/474309

Change 474321 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] role: add aggregations for TCP Fast Open to prometheus global

https://gerrit.wikimedia.org/r/474321

Change 473295 merged by Cwhite:
[operations/puppet@production] ntp: ensure absent ntpd diamond collector

https://gerrit.wikimedia.org/r/473295

Change 474280 merged by Muehlenhoff:
[operations/puppet@production] Remove Diamond from Swift backends

https://gerrit.wikimedia.org/r/474280

Change 474695 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Disable Diamond on WDQS hosts

https://gerrit.wikimedia.org/r/474695

Change 474698 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Absent unused Diamond collector for ldap/corp

https://gerrit.wikimedia.org/r/474698

Change 474699 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove Diamond from openldap/corp servers

https://gerrit.wikimedia.org/r/474699

Change 474321 merged by Cwhite:
[operations/puppet@production] role: add aggregations for TCP Fast Open to prometheus global

https://gerrit.wikimedia.org/r/474321

Change 474698 merged by Muehlenhoff:
[operations/puppet@production] Absent unused Diamond collector for ldap/corp

https://gerrit.wikimedia.org/r/474698

Change 474699 merged by Muehlenhoff:
[operations/puppet@production] Remove Diamond from openldap/corp servers

https://gerrit.wikimedia.org/r/474699

Change 474695 merged by Muehlenhoff:
[operations/puppet@production] Disable Diamond on WDQS hosts

https://gerrit.wikimedia.org/r/474695

Change 474855 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove server-board

https://gerrit.wikimedia.org/r/474855

Change 474864 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove ApacheStatusSimpleCollector

https://gerrit.wikimedia.org/r/474864

Change 474865 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove PyBalStateCollector

https://gerrit.wikimedia.org/r/474865

Change 474887 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove absented Diamond collector for NTP

https://gerrit.wikimedia.org/r/474887

Change 474855 merged by Muehlenhoff:
[operations/puppet@production] Remove server-board

https://gerrit.wikimedia.org/r/474855

Change 474922 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Disable Diamond on Graphite hosts

https://gerrit.wikimedia.org/r/474922

Change 474930 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove Diamond from restbase servers

https://gerrit.wikimedia.org/r/474930

Change 474309 merged by jenkins-bot:
[operations/puppet/nginx@master] nginx: remove diamond::collector::nginx reference

https://gerrit.wikimedia.org/r/474309

Change 474940 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] nginx: use latest commit

https://gerrit.wikimedia.org/r/474940

Change 474887 merged by Cwhite:
[operations/puppet@production] Remove absented Diamond collector for NTP

https://gerrit.wikimedia.org/r/474887

Change 474865 merged by Muehlenhoff:
[operations/puppet@production] Remove PyBalStateCollector

https://gerrit.wikimedia.org/r/474865

Change 474864 merged by Muehlenhoff:
[operations/puppet@production] Remove ApacheStatusSimpleCollector

https://gerrit.wikimedia.org/r/474864

Change 475009 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] mw_rc_irc: ensure diamond::collector absent

https://gerrit.wikimedia.org/r/475009

Change 475010 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] mw_rc_irc: remove diamond::collector resource

https://gerrit.wikimedia.org/r/475010

colewhite updated the task description. (Show Details)Nov 20 2018, 10:48 PM

Change 474940 merged by Cwhite:
[operations/puppet@production] nginx: use latest commit

https://gerrit.wikimedia.org/r/474940

colewhite updated the task description. (Show Details)Nov 21 2018, 4:20 PM

Change 474922 merged by Muehlenhoff:
[operations/puppet@production] Disable Diamond on Graphite hosts

https://gerrit.wikimedia.org/r/474922

Change 474930 merged by Muehlenhoff:
[operations/puppet@production] Remove Diamond from restbase servers

https://gerrit.wikimedia.org/r/474930

Change 475444 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove Diamond from several analytics roles

https://gerrit.wikimedia.org/r/475444

Change 475444 merged by Muehlenhoff:
[operations/puppet@production] Remove Diamond from several analytics roles

https://gerrit.wikimedia.org/r/475444

Change 475456 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove Diamond from Ganeti hosts

https://gerrit.wikimedia.org/r/475456

Change 475456 merged by Muehlenhoff:
[operations/puppet@production] Remove Diamond from Ganeti hosts

https://gerrit.wikimedia.org/r/475456

Krinkle removed a subscriber: Krinkle.Nov 26 2018, 6:08 PM

Change 475009 merged by Cwhite:
[operations/puppet@production] mw_rc_irc: ensure diamond::collector absent

https://gerrit.wikimedia.org/r/475009

Change 475957 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove IRCDStats collector

https://gerrit.wikimedia.org/r/475957

Change 475967 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Absent Redis Diamond collector on Redis slaves

https://gerrit.wikimedia.org/r/475967

Change 475969 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Absent Redis Diamond collector on Redis masters

https://gerrit.wikimedia.org/r/475969

Change 467264 merged by Muehlenhoff:
[operations/puppet@production] Remove Diamond from DB roles

https://gerrit.wikimedia.org/r/467264

Change 475957 merged by Muehlenhoff:
[operations/puppet@production] Remove IRCDStats collector

https://gerrit.wikimedia.org/r/475957

Change 475010 abandoned by Cwhite:
mw_rc_irc: remove diamond::collector resource and collector script

Reason:
Removed in I44ea7b94ab1eab45a923917dd6f0c5b6ae629c5c

https://gerrit.wikimedia.org/r/475010

Change 476001 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove Diamond on additional DB roles

https://gerrit.wikimedia.org/r/476001

Change 475967 merged by Muehlenhoff:
[operations/puppet@production] Absent Redis Diamond collector on Redis slaves

https://gerrit.wikimedia.org/r/475967

Change 476001 merged by Muehlenhoff:
[operations/puppet@production] Remove Diamond on additional DB roles

https://gerrit.wikimedia.org/r/476001

Change 475969 merged by Muehlenhoff:
[operations/puppet@production] Absent Redis Diamond collector on Redis masters

https://gerrit.wikimedia.org/r/475969

Change 476226 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove Diamond from redis::misc systems

https://gerrit.wikimedia.org/r/476226

Change 476226 merged by Muehlenhoff:
[operations/puppet@production] Remove Diamond from redis::misc systems

https://gerrit.wikimedia.org/r/476226

Change 476606 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Absent NfsdCollector Diamond collector

https://gerrit.wikimedia.org/r/476606

Change 476606 merged by GTirloni:
[operations/puppet@production] Absent NfsdCollector Diamond collector

https://gerrit.wikimedia.org/r/476606

GTirloni updated the task description. (Show Details)Nov 29 2018, 7:54 PM

Change 476883 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Absent Redis Diamond collector on mc* servers

https://gerrit.wikimedia.org/r/476883

Change 476883 merged by Muehlenhoff:
[operations/puppet@production] Absent Redis Diamond collector on mc* servers

https://gerrit.wikimedia.org/r/476883

MoritzMuehlenhoff updated the task description. (Show Details)

Change 477231 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove Redis collector

https://gerrit.wikimedia.org/r/477231

Change 477231 merged by Muehlenhoff:
[operations/puppet@production] Remove Redis collector

https://gerrit.wikimedia.org/r/477231

Krinkle added a subscriber: Krinkle.Dec 4 2018, 2:41 PM

I've deleted the dashboard now. It was created this during an incident investigation and hasn't been used since afaik.

Change 477748 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove Diamond on maps servers

https://gerrit.wikimedia.org/r/477748

Change 477748 merged by Muehlenhoff:
[operations/puppet@production] Remove Diamond on maps servers

https://gerrit.wikimedia.org/r/477748

Change 478016 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove Diamond from DNS roles

https://gerrit.wikimedia.org/r/478016

Change 478368 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] remove directorysize diamond collector

https://gerrit.wikimedia.org/r/478368

Change 478371 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] remove diamond::collector reference from role::labs::nfs::secondary

https://gerrit.wikimedia.org/r/478371

Change 478368 merged by GTirloni:
[operations/puppet@production] remove directorysize diamond collector

https://gerrit.wikimedia.org/r/478368

Change 478016 merged by Muehlenhoff:
[operations/puppet@production] Remove Diamond from DNS roles

https://gerrit.wikimedia.org/r/478016

colewhite updated the task description. (Show Details)Dec 10 2018, 4:20 PM

Change 478371 merged by GTirloni:
[operations/puppet@production] remove diamond::collector reference from role::labs::nfs::secondary

https://gerrit.wikimedia.org/r/478371

Change 469250 merged by Cwhite:
[operations/puppet@production] memcached: remove memcached diamond collector

https://gerrit.wikimedia.org/r/469250

Change 469258 merged by Cwhite:
[operations/puppet@production] memcached: remove diamond::collector resource

https://gerrit.wikimedia.org/r/469258

Change 466907 merged by Cwhite:
[operations/puppet@production] hiera: remove diamond from thumbor role

https://gerrit.wikimedia.org/r/466907

Change 429221 abandoned by Cwhite:
memcached: deprecate Diamond collector

Reason:
Covered by I4506039ff403bce1c5fc5b2093861a937d84e31b

https://gerrit.wikimedia.org/r/429221

colewhite updated the task description. (Show Details)Dec 11 2018, 7:33 PM

Change 479169 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove now obsolete Diamond collector and related conffiles

https://gerrit.wikimedia.org/r/479169

Change 479189 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove Diamond from ORES hosts

https://gerrit.wikimedia.org/r/479189

Change 479169 merged by Muehlenhoff:
[operations/puppet@production] Remove now obsolete Diamond collector and related conffile

https://gerrit.wikimedia.org/r/479169

Change 479189 merged by Muehlenhoff:
[operations/puppet@production] Remove Diamond from ORES hosts

https://gerrit.wikimedia.org/r/479189

Change 479446 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove Diamond from further roles

https://gerrit.wikimedia.org/r/479446

Change 479446 merged by Muehlenhoff:
[operations/puppet@production] Remove Diamond from further roles

https://gerrit.wikimedia.org/r/479446

Change 480032 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove Diamond from further roles

https://gerrit.wikimedia.org/r/480032

Change 480032 merged by Muehlenhoff:
[operations/puppet@production] Remove Diamond from further roles

https://gerrit.wikimedia.org/r/480032

MoritzMuehlenhoff closed this task as Resolved.Dec 18 2018, 4:18 PM

This is done (and the task is getting too big as well), I created T212231 for some followup work.

Change 466908 merged by Cwhite:
[operations/puppet@production] hiera: remove diamond from labweb role

https://gerrit.wikimedia.org/r/466908

Change 464366 abandoned by Cwhite:
memcached, redis: remove diamond

Reason:
no longer needed

https://gerrit.wikimedia.org/r/464366