Since Dec 12 deploy the memory usage in ChangeProp started to grow over time: https://grafana.wikimedia.org/dashboard/db/eventbus?from=1481464969163&to=1481730678921&var-site=All&var-topic=All
and there's been a couple of worker restarts already. The deploy was big and contained the following changes:
Update change-propagation to b2bf30d List of changes: fdf1267 Use delivery callback to guarantee delivery 3717c06 Use 'connect' callback to figure out we're connected. Better then ready event 639eb6f Don't fail if the resolver is not found for delivery report ad6d4b9 Actually set up the logger 9e940c4 Got rid of the function keyword cf9a705 Use shared kafka management tools b40f5ed Don't clean in travis 373b9e5 Run scripts after npm install 615b2aa Better sourcing the file 875151a Better sourcing the file 2cbaa03 Better check for travis environment 3f525cb Correctly kill kafka afterwards f95bf71 Improved cleanup script soursing 35f7b9a Improved CPU usage b2bf30d Release v0.6.3 xxxxxxx Update node module dependencies
I suspect that memory leak is related to the introduction of delivery callbacks, but this needs more investigation.