Page MenuHomePhabricator
Create Task
Maniphest T141524

eventbus should send statsd in batches
Closed, DeclinedPublic
Actions

Description

I noticed eventbus/eventlogging doesn't seem to send statsd metrics in batches even though it should, up until 512 bytes per UDP packet sent

@ ......H5.eventbus.counters.eventlogging.service.EventHandler.POST.201:1|c
14:14:17.802493 IP kafka1001.eqiad.wmnet.55468 > graphite1001.eqiad.wmnet.8125: UDP, length 75
E..g >@.@.."
@..
@ ......S5.eventbus.timers.eventlogging.service.EventHandler.POST.201:1.85799598694|ms
14:14:17.802773 IP kafka1001.eqiad.wmnet.55468 > graphite1001.eqiad.wmnet.8125: UDP, length 64
E..\ ?@.@..,
@..
@ ......H5.eventbus.counters.eventlogging.service.EventHandler.POST.201:1|c
14:14:17.808752 IP kafka1001.eqiad.wmnet.55468 > graphite1001.eqiad.wmnet.8125: UDP, length 75
E..g A@.@...
@..
@ ......S5.eventbus.timers.eventlogging.service.EventHandler.POST.201:2.11000442505|ms
14:14:17.809090 IP kafka1001.eqiad.wmnet.55468 > graphite1001.eqiad.wmnet.8125: UDP, length 64
E..\ B@.@..)
@..
@ ......H5.eventbus.counters.eventlogging.service.EventHandler.POST.201:1|c
14:14:17.816720 IP kafka1001.eqiad.wmnet.55468 > graphite1001.eqiad.wmnet.8125: UDP, length 75
E..g C@.@...
@..
@ ......S5.eventbus.timers.eventlogging.service.EventHandler.POST.201:4.09197807312|ms
14:14:17.817030 IP kafka1001.eqiad.wmnet.55468 > graphite1001.eqiad.wmnet.8125: UDP, length 64
E..\ D@.@..'
@..

it looks like eventbus is using statsd from pypi, using a Pipeline should do the trick: https://github.com/jsocol/pystatsd/blob/master/docs/pipeline.rst

Related Objects
Search...

Event Timeline

fgiunchedi created this task.Jul 28 2016, 2:22 PM
Ottomata claimed this task.Jul 28 2016, 4:53 PM
Ottomata added a comment.Jul 28 2016, 5:42 PM

So, these stats are HTTP request statistics directly from Python tornado eventlogging-service (eventbus) uses tornado as its http server. The metrics are collected by sprockets metrics and sent to statsd.

Unfortunetly, it looks like they send directly to the socket, instead of the python statsd client.

https://github.com/sprockets/sprockets.clients.statsd/blob/master/sprockets/clients/statsd/__init__.py

Maybe we can upstream a patch? If not, we can fork. We build our own python .deb packages used by eventbus for this.

• Pchelolo added a project: Services (watching).Oct 13 2016, 12:05 AM
Krinkle removed a project: Grafana.Nov 18 2016, 6:24 PM
• mobrovac added a project: Platform Team Legacy (Watching / External).Dec 20 2018, 12:03 PM
Restricted Application added a project: Analytics. · View Herald TranscriptDec 20 2018, 12:03 PM
Milimetric closed this task as Declined.Jan 7 2019, 4:55 PM
Milimetric subscribed.

won't fix this because we're working on a new implementation

• Pchelolo subscribed.Jan 7 2019, 4:56 PM

And the new implementation is based on service-runner which batch stats by default.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL