When emery got decomissioned, erbium around mid-April 2014 had it's
udp2log instance turned from a multicast consumer into a consumer of
only varnishncsa. Hence, since then, erbium only receives logs for the
traffic the ssl terminators cause on the caches, but not the logs for
traffic arriving at the ssl terminators.
However, to correctly count ssl traffic, webstatscollector's filter
(which runs on erbium) relies on seeing the logs at the ssl terminators.
So as a result, ssl traffic was not counted in webstatscollector's
output since mid-April 2014.
Discussing a bit with ottomata, it seems the best way forward [1] [2]
is to try moving webstatscollector's filter to a host that still
consumes the udp2log multicast.
oxygen consumes the udp2log multicast, and does not look overloaded,
so we could try moving webstatscollector's filter there.
[1] We could instead turn multicast consumption on erbium on again. But
that would also affect all the other udp2log filters that meanwhile got
added to erbium. So it would cause further issues.
[2] We could update webstatscollector to stop dropping local http
requests from the ssl terminators. That would be nice as it would mean
that the overall picture of the webstatscollector pipeline gets
simpler. However, people are afraid to touch webstatscollector code,
and we want to phase it out anyways.
Version: unspecified
Severity: normal
Whiteboard: u=Community c=General/Unknown p=0 s=2014-06-26