Page MenuHomePhabricator
Create Task
Maniphest T126447

add more statsdlb instances for more throughput
Closed, ResolvedPublic
Actions

Description

graphite1001 has cpu to spare, and we're currently running only one statsdlb process which often spikes to using one core at 100%, with SO_REUSEADDR already in statsdlb we should be able to have multiple processes all receiving incoming udp traffic and share the load

Details

SubjectRepoBranchLines +/-
Add a new statsd_proxy module and replace statsdlboperations/puppetproduction+80 -8
use SO_REUSEPORT in code, not SO_REUSEADDRoperations/software/statsdlbmaster+8 -2
set SO_REUSEADDR before bind()operations/software/statsdlbmaster+7 -5
Customize query in gerrit

Related Objects
Search...

Event Timeline

fgiunchedi created this task.Feb 10 2016, 11:59 AM
fgiunchedi claimed this task.
fgiunchedi raised the priority of this task from to Medium.
fgiunchedi updated the task description. (Show Details)
fgiunchedi added projects: SRE, Grafana.
fgiunchedi added subscribers: ori, fgiunchedi.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 10 2016, 11:59 AM
gerritbot subscribed.Feb 29 2016, 4:10 PM

Change 273920 had a related patch set uploaded (by Filippo Giunchedi):
set SO_REUSEADDR before bind()

https://gerrit.wikimedia.org/r/273920

gerritbot added a project: Patch-For-Review.Feb 29 2016, 4:10 PM
gerritbot added a comment.Feb 29 2016, 9:24 PM

Change 273920 merged by Ori.livneh:
set SO_REUSEADDR before bind()

https://gerrit.wikimedia.org/r/273920

gerritbot added a comment.Apr 8 2016, 1:18 PM

Change 282356 had a related patch set uploaded (by Faidon Liambotis):
Add a new statsd_proxy module and replace statsdlb

https://gerrit.wikimedia.org/r/282356

gerritbot added a comment.Jul 1 2016, 2:53 PM

Change 296920 had a related patch set uploaded (by Filippo Giunchedi):
use SO_REUSEPORT in code, not SO_REUSEADDR

https://gerrit.wikimedia.org/r/296920

gerritbot added a comment.Jul 4 2016, 11:04 AM

Change 296920 merged by Filippo Giunchedi:
use SO_REUSEPORT in code, not SO_REUSEADDR

https://gerrit.wikimedia.org/r/296920

gerritbot added a comment.Jul 28 2016, 10:55 AM

Change 282356 merged by Filippo Giunchedi:
Add a new statsd_proxy module and replace statsdlb

https://gerrit.wikimedia.org/r/282356

fgiunchedi mentioned this in rOPUP52527c28eace: Add a new statsd_proxy module and replace statsdlb.Jul 28 2016, 10:57 AM
fgiunchedi closed this task as Resolved.Jul 28 2016, 12:21 PM

Resolving this since it essentially happened by replacing statsdlb with statsd_proxy, followup on T101141

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL