Change 817307 had a related patch set uploaded (by Jbond; author: jbond):

[operations/puppet@production] P:base::firewall: Add requestctl definitions to ferm

https://gerrit.wikimedia.org/r/817307

Change 817307 merged by Jbond:

[operations/puppet@production] P:base::firewall: Add requestctl definitions to ferm

https://gerrit.wikimedia.org/r/817307

Change 822091 had a related patch set uploaded (by Jbond; author: jbond):

[operations/puppet@production] P:firewall: fix template errors

https://gerrit.wikimedia.org/r/822091

Change 822091 merged by Jbond:

[operations/puppet@production] P:firewall: fix template errors

https://gerrit.wikimedia.org/r/822091

Change 822106 had a related patch set uploaded (by Jbond; author: jbond):

[operations/puppet@production] P:sretest: test behaviour of empty define

https://gerrit.wikimedia.org/r/822106

Change 822106 merged by Jbond:

[operations/puppet@production] P:sretest: test behaviour of empty define

https://gerrit.wikimedia.org/r/822106

Change 822125 had a related patch set uploaded (by Jbond; author: John Bond):

[operations/puppet@production] P:base::firewall: use either etc or abuse_nets

https://gerrit.wikimedia.org/r/822125

Change 822125 merged by Jbond:

[operations/puppet@production] P:base::firewall: use either etc or abuse_nets

https://gerrit.wikimedia.org/r/822125

Change 822340 had a related patch set uploaded (by Jbond; author: jbond):

[operations/puppet@production] O:sretest: enable confd based abuse filter on sretest

https://gerrit.wikimedia.org/r/822340

Change 822340 merged by Jbond:

[operations/puppet@production] O:sretest: enable confd based abuse filter on sretest

https://gerrit.wikimedia.org/r/822340

Change 822361 had a related patch set uploaded (by Jbond; author: jbond):

[operations/puppet@production] C:ferm: allow useres to read config files, needed for nrpe

https://gerrit.wikimedia.org/r/822361

Change 822361 merged by Jbond:

[operations/puppet@production] C:ferm: allow useres to read config files, needed for nrpe

https://gerrit.wikimedia.org/r/822361

In my brief testing it appears that ferm is happy to accept bogus IPs and try to load them in with iptables, leaving the box with no rules at all. @jbond how confident are we that etcd will never have bogus data?

Change 823608 had a related patch set uploaded (by Jbond; author: jbond):

[operations/software/conftool@master] reqconfig: add ip validation for ipblocks

https://gerrit.wikimedia.org/r/823608

In my brief testing it appears that ferm is happy to accept bogus IPs

indeed and this also affects the ferm::rule resource. i.e. a typo in a ferm::rule could cause the entire firewall rule to de dropped. We do at least get an icinga alert for this as it puts the ferm status in a bad state.

Looking at the output from the ferm systemd unit it seems like it tries to fall back to using iptables-restore if there is an error however this also fails

twork `foobar' not found
Aug 16 12:00:03 sretest1001 ferm[24432]: Error occurred at line: 27
Aug 16 12:00:03 sretest1001 ferm[24432]: Try `iptables-restore -h' or 'iptables-restore --help' for more information.
Aug 16 12:00:03 sretest1001 ferm[24432]: Failed to run /sbin/iptables-restore
Aug 16 12:00:03 sretest1001 ferm[24432]: ip6tables-restore v1.8.2 (nf_tables): host/network `foobar' not found
Aug 16 12:00:03 sretest1001 ferm[24432]: Error occurred at line: 21
Aug 16 12:00:03 sretest1001 ferm[24432]: Try `ip6tables-restore -h' or 'ip6tables-restore --help' for more information.
Aug 16 12:00:03 sretest1001 ferm[24432]: Failed to run /sbin/ip6tables-restore
Aug 16 12:00:03 sretest1001 ferm[24432]: Firewall rules rolled back.
Aug 16 12:00:03 sretest1001 ferm[24432]:  failed!

Ill create a separate task to track this issue as its a bit more generic.

how confident are we that etcd will never have bogus data?

As far as i can tell we dont do any validation, i have created a PS and will speak with joe to see if we can get it added so we validate the date before adding it to confd

Change 823616 had a related patch set uploaded (by Jbond; author: jbond):

[operations/puppet@production] P:base::firewall: use reload instead of restart

https://gerrit.wikimedia.org/r/823616

In my brief testing it appears that ferm is happy to accept bogus IPs

Worth noting that if you use systemd reload ferm instead of restart then it at least leaves iptables with the last none good set of rules. if have corrected the confd::file resource to use reload instead of restart

Change 823616 merged by Jbond:

[operations/puppet@production] P:base::firewall: use reload instead of restart

https://gerrit.wikimedia.org/r/823616

how confident are we that etcd will never have bogus data?

As far as i can tell we dont do any validation, i have created a PS and will speak with joe to see if we can get it added so we validate the date before adding it to confd

great, thanks!

In T313825#8157069, @jbond wrote:

In my brief testing it appears that ferm is happy to accept bogus IPs

Worth noting that if you use systemd reload ferm instead of restart then it at least leaves iptables with the last none good set of rules. if have corrected the confd::file resource to use reload instead of restart

very nice, that seems like much saner behavior.

Change 823608 merged by jenkins-bot:

[operations/software/conftool@master] reqconfig: add ip validation for ipblocks

https://gerrit.wikimedia.org/r/823608

Change 883878 had a related patch set uploaded (by Jbond; author: John Bond):

[operations/dns@master] wikimedia.org: add cond SRV records

https://gerrit.wikimedia.org/r/883878

Change 883878 merged by Jbond:

[operations/dns@master] wikimedia.org: add cond SRV records

https://gerrit.wikimedia.org/r/883878

Change 883888 had a related patch set uploaded (by Jbond; author: Jbond):

[operations/puppet@production] firewall: Add requestctl support to ferm globaly

https://gerrit.wikimedia.org/r/883888

Change 883888 merged by Jbond:

[operations/puppet@production] firewall: Add requestctl support to ferm globaly

https://gerrit.wikimedia.org/r/883888

Change 883891 had a related patch set uploaded (by Jbond; author: Jbond):

[operations/puppet@production] firewall: Add requestctl support to ferm globaly

https://gerrit.wikimedia.org/r/883891

Change 883891 merged by Jbond:

[operations/puppet@production] firewall: Add requestctl support to ferm globaly

https://gerrit.wikimedia.org/r/883891

Change 883893 had a related patch set uploaded (by Jbond; author: Jbond):

[operations/puppet@production] firewall: Add requestctl support to ferm globaly

https://gerrit.wikimedia.org/r/883893

Change 883973 had a related patch set uploaded (by Jbond; author: John Bond):

[operations/puppet@production] confd::file: allow to specify fully qualified prefix

https://gerrit.wikimedia.org/r/883973

Change 883973 merged by Jbond:

[operations/puppet@production] confd::file: allow to specify fully qualified prefix

https://gerrit.wikimedia.org/r/883973

Change 883893 merged by Jbond:

[operations/puppet@production] firewall: Add requestctl support to ferm globaly

https://gerrit.wikimedia.org/r/883893