Page MenuHomePhabricator
Create Task
Maniphest T177195

Reduce technical debt in metrics monitoring
Closed, ResolvedPublic
Actions

Description

Increase porting of statsd/graphite metrics to Prometheus

  • Port non-deprecated Diamond collectors to Prometheus, making Diamond deprecated in production by EOQ
  • Export Prometheus-compatible JVM metrics from JVMs in production (stretch)
  • Add Prometheus client support for varnish/statsd metrics daemons

Details

ProjectBranchLines +/-Subject
operations/puppetproduction+7 -0prometheus: aggregate availability for varnish backends
operations/puppetproduction+13 -10prometheus: tweak varnish aggregation rules
operations/puppetproduction+7 -7prometheus: calculate nginx/varnish availability over five minutes
operations/puppetproduction+18 -1prometheus: aggregate nginx requests and availability
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedfgiunchediT177195 Reduce technical debt in metrics monitoring
 ResolvedfgiunchediT177196 Port non-deprecated Diamond collectors to Prometheus
 ResolvedfgiunchediT148637 Port redis statistics to Prometheus
 ResolvedakosiarisT179306 Port postgresql metrics to Prometheus
 DeclinedGehelT175799 port elasticsearch diamond collector to prometheus
 ResolvedfgiunchediT179302 prometheus metrics and grafana dashboard for exim
 ResolvedfgiunchediT179565 Port exim statistics to Prometheus
 ResolvedfgiunchediT181794 Have jenkins run mtail tests via tox/nose
 ResolvedMoritzMuehlenhoffT181511 Package openldap collector for Prometheus and adapt metrics
 ResolvedMoritzMuehlenhoffT181620 Package PDNS Recursor collector for Prometheus and adapt metrics
 ResolvedGehelT181627 Port elasticsearch metrics to Prometheus
 ResolvedMoritzMuehlenhoffT181802 Package RabbitMQ exporter for Prometheus and adapt metrics
 ResolvedfgiunchediT181995 Port nutcracker statistics to Prometheus
 ResolvedMoritzMuehlenhoffT182095 Create Prometheus exporter for Etherpad
 ResolvedMoritzMuehlenhoffT182196 Create ircd exporter for Prometheus
 ResolvedMoritzMuehlenhoffT182773 Create Prometheus exporter for wdqs-updater
 ResolvedMoritzMuehlenhoffT182857 Create Prometheus exporter for Blazegraph
 ResolvedMoritzMuehlenhoffT182970 Create Prometheus exporter for PowerDNS
 OpenNoneT187434 Include apache_exporter in puppet module httpd (was: apache)
 Resolvedbd808T211684 Toolforge: Port sge.py stats to Prometheus
 ResolvedBstormT215845 Add monitoring for disabled grid nodes to the prometheus collector
 OpenNoneT213567 Toolforge: refresh grafana dashboard
 OpenNoneT177197 Export Prometheus-compatible JVM metrics from JVMs in production
 ResolvedelukeyT181728 Stop using jmx_exporter deployed via scap in favour of Debian package
 OpenNoneT182759 Add Prometheus exporter to Jenkins instances
 OpenNoneT184086 Add prometheus exporter to Gerrit
 OpenNoneT192948 Upgrade prometheus-jmx-exporter on all services using it
 ResolvedcolewhiteT276595 Upgrade prometheus-jmx-exporter
 ResolvedfgiunchediT177199 Add Prometheus client support for varnish/statsd metrics daemons
 ResolvedfgiunchediT183926 Limit http methods reported by varnishmtail
 ResolvedfgiunchediT184942 Deprecate python varnish cachestats
 ResolvedKrinkleT190978 Update ResourceLoader dashboard to query varnishrls data from Prometheus instead
 ResolvedDzahnT177225 Uninstall ganglia from the fleet
 ResolvedDzahnT183873 Update ganglia mentions in prominent documentation
 ResolvedAndrewT183917 remove cloud VPS project 'ganglia'

Event Timeline

fgiunchedi created this task.Oct 2 2017, 9:34 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 2 2017, 9:34 AM
fgiunchedi created subtask T177196: Port non-deprecated Diamond collectors to Prometheus.Oct 2 2017, 9:37 AM
fgiunchedi created subtask T177197: Export Prometheus-compatible JVM metrics from JVMs in production.
fgiunchedi created subtask T177199: Add Prometheus client support for varnish/statsd metrics daemons.Oct 2 2017, 9:43 AM
Aklapper added a project: Technical-Debt.Oct 2 2017, 1:59 PM
fgiunchedi added a project: User-fgiunchedi.Oct 4 2017, 8:11 AM
fgiunchedi moved this task from Backlog to Doing on the User-fgiunchedi board.Oct 11 2017, 12:48 PM
Dzahn added a subtask: T177225: Uninstall ganglia from the fleet.Oct 21 2017, 3:29 PM
MoritzMuehlenhoff triaged this task as High priority.Nov 7 2017, 11:11 AM
Dzahn closed subtask T177225: Uninstall ganglia from the fleet as Resolved.Dec 20 2017, 3:57 AM
fgiunchedi closed subtask T177199: Add Prometheus client support for varnish/statsd metrics daemons as Resolved.Jan 15 2018, 5:49 PM
gerritbot added a subscriber: gerritbot.Feb 1 2018, 3:15 PM

Change 407445 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: aggregate nginx requests and availability

https://gerrit.wikimedia.org/r/407445

gerritbot added a project: Patch-For-Review.Feb 1 2018, 3:15 PM
gerritbot added a comment.Feb 1 2018, 3:43 PM

Change 407445 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: aggregate nginx requests and availability

https://gerrit.wikimedia.org/r/407445

gerritbot added a comment.Feb 5 2018, 2:53 PM

Change 408274 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: calculate nginx/varnish availability over five minutes

https://gerrit.wikimedia.org/r/408274

gerritbot added a comment.Feb 5 2018, 2:55 PM

Change 408274 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: calculate nginx/varnish availability over five minutes

https://gerrit.wikimedia.org/r/408274

gerritbot added a comment.Feb 12 2018, 9:21 AM

Change 409820 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: tweak varnish aggregation rules

https://gerrit.wikimedia.org/r/409820

gerritbot added a comment.Feb 12 2018, 9:30 AM

Change 409820 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: tweak varnish aggregation rules

https://gerrit.wikimedia.org/r/409820

gerritbot added a comment.Feb 14 2018, 8:51 AM

Change 410402 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: aggregate availability for varnish backends

https://gerrit.wikimedia.org/r/410402

gerritbot added a comment.Feb 14 2018, 8:55 AM

Change 410402 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: aggregate availability for varnish backends

https://gerrit.wikimedia.org/r/410402

Liuxinyu970226 added a subscriber: Liuxinyu970226.May 14 2018, 1:52 AM
fgiunchedi moved this task from Doing to Up next on the User-fgiunchedi board.Jul 11 2018, 1:50 PM
CDanis added a project: User-CDanis.Nov 9 2018, 7:28 PM
CDanis moved this task from Backlog to Radar on the User-CDanis board.
ArielGlenn removed a project: Patch-For-Review.Nov 13 2018, 9:50 AM
• Phabricator_maintenance moved this task from Backlog to Acknowledged on the SRE board.Jan 26 2019, 9:29 PM
fgiunchedi closed subtask T177196: Port non-deprecated Diamond collectors to Prometheus as Resolved.Feb 5 2019, 8:43 AM
fgiunchedi added a project: observability.Jul 24 2019, 9:46 AM
fgiunchedi closed this task as Resolved.Jun 3 2020, 12:58 PM
fgiunchedi claimed this task.

Boldly resolving, the bulk of the work here has been done and the remaining tasks are low(er) priority

Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL