Page MenuHomePhabricator
Create Task
Maniphest T179565

Port exim statistics to Prometheus
Closed, ResolvedPublic
Actions

Description

The goal is to have exim statistics exported as Prometheus metrics by looking at exim logs with mtail.

Details

ProjectBranchLines +/-Subject
operations/puppetproduction+18 -1prometheus: add mtail/exim jobs
operations/puppetproduction+9 -0role: add ferm rule for mtail on mx
operations/puppetproduction+3 -1mtail: allow changing running group
operations/puppetproduction+114 -0mtail: add test scaffolding
operations/puppetproduction+111 -0mx: export metrics from exim4 mainlog
Customize query in gerrit

Related Objects
Search...

Event Timeline

fgiunchedi created this task.Nov 2 2017, 9:24 AM
gerritbot added a comment.Nov 2 2017, 11:46 AM

Change 388032 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] mx: export metrics from exim4 mainlog

https://gerrit.wikimedia.org/r/388032

gerritbot added a project: Patch-For-Review.Nov 2 2017, 11:46 AM
fgiunchedi mentioned this in T177196: Port non-deprecated Diamond collectors to Prometheus.Nov 2 2017, 1:58 PM
fgiunchedi moved this task from Backlog to Doing on the User-fgiunchedi board.
gerritbot added a comment.Nov 3 2017, 3:56 PM

Change 388478 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] mtail: add test scaffolding

https://gerrit.wikimedia.org/r/388478

MoritzMuehlenhoff triaged this task as High priority.Nov 13 2017, 7:34 AM
gerritbot added a comment.Nov 13 2017, 11:27 AM

Change 388032 merged by Filippo Giunchedi:
[operations/puppet@production] mx: export metrics from exim4 mainlog

https://gerrit.wikimedia.org/r/388032

gerritbot added a comment.Nov 13 2017, 12:21 PM

Change 390982 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] mtail: allow changing running group

https://gerrit.wikimedia.org/r/390982

gerritbot added a comment.Nov 13 2017, 12:27 PM

Change 388478 merged by Filippo Giunchedi:
[operations/puppet@production] mtail: add test scaffolding

https://gerrit.wikimedia.org/r/388478

gerritbot added a comment.Nov 13 2017, 12:33 PM

Change 390982 merged by Filippo Giunchedi:
[operations/puppet@production] mtail: allow changing running group

https://gerrit.wikimedia.org/r/390982

gerritbot added a comment.Nov 13 2017, 2:44 PM

Change 391011 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] role: add ferm rule for mtail on mx

https://gerrit.wikimedia.org/r/391011

gerritbot added a comment.Nov 15 2017, 4:11 PM

Change 391011 merged by Filippo Giunchedi:
[operations/puppet@production] role: add ferm rule for mtail on mx

https://gerrit.wikimedia.org/r/391011

gerritbot added a comment.Nov 17 2017, 1:15 PM

Change 392039 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] prometheus: add mtail/exim jobs

https://gerrit.wikimedia.org/r/392039

gerritbot added a comment.Nov 20 2017, 11:32 AM

Change 392039 merged by Filippo Giunchedi:
[operations/puppet@production] prometheus: add mtail/exim jobs

https://gerrit.wikimedia.org/r/392039

hashar mentioned this in T180935: Various puppet issues in deployment-prep.Nov 21 2017, 8:54 AM
fgiunchedi created subtask T181794: Have jenkins run mtail tests via tox/nose.Dec 1 2017, 11:41 AM
fgiunchedi closed this task as Resolved.Dec 5 2017, 10:48 AM
fgiunchedi claimed this task.
fgiunchedi added subscribers: herron, faidon.

Metrics are being collected, I started a dashboard at https://grafana.wikimedia.org/dashboard/db/mail (though improvements are welcome of course!) and I'm resolving the task (cc @herron @faidon as mail people)

Dzahn added a subscriber: Dzahn.Dec 13 2017, 12:34 AM

removed ganglia from mx1001/mx2001

Dzahn mentioned this in T179302: prometheus metrics and grafana dashboard for exim.Dec 13 2017, 6:41 PM
fgiunchedi closed subtask T181794: Have jenkins run mtail tests via tox/nose as Resolved.Dec 14 2017, 6:02 PM
fgiunchedi mentioned this in T71072: Monitor mail system in Graphite.Dec 21 2017, 11:22 AM
MoritzMuehlenhoff mentioned this in T210991: Deprecate Diamond collectors in Tool Labs / Tool Forge.Dec 3 2018, 10:01 AM
bd808 mentioned this in T210993: Deprecate Diamond collectors in Cloud VPS.Jan 1 2020, 9:46 PM
Cyrille37 added a subscriber: Cyrille37.Jun 12 2020, 7:16 PM

Hi,

Please, where is the source code of this exim4 to prometheus ?

There are very few exim4 exporter findable on the net, perhaps yours is the greater :-)

Thanks & cheers, Cyrille37.

fgiunchedi added a comment.Jun 15 2020, 8:46 AM

Hi @Cyrille37,
we're currently using mtail (https://github.com/google/mtail) to turn exim logs into metrics. See also https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/production/modules/mtail/files/programs/exim.mtail for the current configuration

Hope this helps!
Filippo

Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL