We have seen several varnish-fe crashes with the following errors:
Jan 08 15:05:40 cp3050 varnishd[3571]: Child (3611) said <jemalloc>: Error in munmap(): Cannot allocate memory
Jan 08 15:06:22 cp3050 varnishd[3571]: Failed to kill child with PID 3611: Operation not permitted
Jan 08 15:06:22 cp3050 varnishd[3571]: Unexpected reply from ping: 400 CLI communication error (hdr)
Jan 08 15:06:22 cp3050 varnishd[3571]: Failed to kill child with PID 3611: Operation not permitted
Jan 08 15:06:22 cp3050 varnishd[3571]: Unexpected reply from ping: 400 CLI communication error
Jan 08 15:06:24 cp3050 varnishd[3571]: Child (3611) died signal=6
Jan 08 15:06:24 cp3050 varnishd[3571]: Child (3611) Panic at: Wed, 08 Jan 2020 15:06:24 GMT
Assert error in VGZ_NewGzip(), cache/cache_gzip.c line 142:
Condition(Z_OK == i) not true.
[...]Note that this is not the same error message as T241593. Also T185968 is different, in that the OOM-killer has nothing to do with the crashes observed during the past days.
Perhaps interestingly, the following text was removed in 2012 from the Varnish troubleshooting guide:
Specifically if you see the "Error in munmap" error on Linux you might want to increase the amount of maps available. Linux is limited to a maximum of 64k maps. Setting vm.max_max_count i sysctl.conf will enable you to increase this limit. You can inspect the number of maps your program is consuming by counting the lines in /proc/$PID/maps.
The actual sysctl setting name is vm.max_map_count.