When new execution nodes are launched, a public IP has to be assigned to that host. Sometimes this is forgotten and then causes bots failing because a large number of them shares an IP which some third-party sites consider suspicious.

As this failure method is not obvious to diagnose, it might be interesting to test in Puppet whether an execution node has been assigned a public IP. This can be done by checking if the fact $::ec2_public_ipv4 is defined and error()ing out if not.