While testing Cumin I discovered an inconsistent state on dbproxy1011, where there are some leftovers of ferm that probably was enabled in the past and was removed at some point. The same happened on multatuli (where it was fixed re-adding the firewall).
It seems that the role applied to dbproxy1011 (::role::mariadb::proxy::master) includes role::mariadb::ferm that in turn defines a ferm::service and a ferm::rule resources that in turn define a virtual file resource each that requires the File['/etc/ferm/conf.d'] resource that in turn requires the Package['ferm'] resource.
But those hosts don't seems to use the ferm class resource, that is the one that realizes the virtual resources into concrete ones, hence do not trigger all this chain of events.
Other dbproxies that I've checked don't have the ferm module installed at all and also looking at a puppet-compiler result for both dbproxy1010 and dbproxy1011 they don't show any ferm package installation, although iptable_filter kernel module is loaded on both of them.
I'm then assuming that the presence of ferm on dbproxy1011 is a leftover of a previous configuration that was including the ferm class that was reverted at some point.
If this is the case, I think that it shows a problem in our firewall configuration because leaving stale applied iptables rules might have weird and unexpected issues. Maybe we should consider or to have a rule to reimage any host that goes from having a firewall to not having it, or to ensure that if not used, the whole firewall dependencies are cleared (ferm pacakge, iptables loaded modules, etc.).
Thoughts?