This task is to track the work to introduce filtering for neutron BGP addresses.
Reported by diffscan: eno2-2120.cloudnet2002-dev.wikimedia.org and eno2-2120.cloudnet2003-dev.wikimedia.org, ports 22 and 9100
| Project | Branch | Lines +/- | Subject | |
|---|---|---|---|---|
| operations/homer/public | master | +39 -0 | Add cloud-out4 firewall filter |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T209460 CloudVPS: network architecture | |||
| Open | None | T244727 CloudVPS: networking improvements | |||
| Stalled | None | T245606 CloudVPS: enable BGP in the neutron transport network | |||
| Resolved | aborrero | T246887 CloudVPS: introduce filtering for neutron BGP addresses |
This is my initial proposal:
diff -u router.org router.new --color --- router.org 2020-03-04 12:45:50.139827978 +0100 +++ router.new 2020-03-04 14:06:26.158971812 +0100 @@ -804,6 +804,7 @@ family inet { filter { input cloud-in4; + output cloud-out4; } address 208.80.153.187/29 { vrrp-group 121 { @@ -3199,6 +3200,34 @@ then accept; } } + /* Applied to cloud instance traffic entering the cloud-instance-transport vlan */ + filter cloud-out4 { + interface-specific; + term cloudnet-accept-bgp { + from { + destination-address { + /* br-external.cloudnet2002-dev.wikimedia.org */ + 208.80.153.188/32; + /* br-external.cloudnet2003-dev.wikimedia.org */ + 208.80.153.189/32; + } + protocol tcp; + destination-port bgp; + } + then accept; + } + term cloudnet-else-drop { + from { + destination-address { + /* br-external.cloudnet2002-dev.wikimedia.org */ + 208.80.153.188/32; + /* br-external.cloudnet2003-dev.wikimedia.org */ + 208.80.153.189/32; + } + } + then discard; + } + } /* Applied to cloud instance traffic going out of the cloud-instance-transport vlan. Last audit in T199435 and T211921. */ filter cloud-in4 { interface-specific;
please @ayounsi review and double-check.
Will push the following to keep the previous behavior and be on a whitelist basis instead of blacklist. We can tune it later on if needed.
[edit firewall family inet]
filter cloud-in4 { ... }
+ /* T246887 */
+ filter cloud-out4 {
+ interface-specific;
+ term allow-local {
+ from {
+ source-address {
+ 208.80.153.184/29;
+ }
+ }
+ then accept;
+ }
+ term allow-cloudnet-gw {
+ from {
+ destination-address {
+ 208.80.153.190/32;
+ }
+ }
+ then accept;
+ }
+ term deny-local {
+ from {
+ destination-address {
+ 208.80.153.184/29;
+ }
+ }
+ then discard;
+ }
+ }Mentioned in SAL (#wikimedia-operations) [2020-03-06T14:44:56Z] <XioNoX> add cloud-out4 firewall filter in codfw - T246887
Confirmed that nc -zv 208.80.153.189 22 doesn't work anymore. While ping to 208.80.153.190 does.
I'll send a Homer CR to make it generic across both sites.
Change 577575 had a related patch set uploaded (by Ayounsi; owner: Ayounsi):
[operations/homer/public@master] Add cloud-out4 firewall filter
This filter might be blocking legit traffic.
I believe we need to explicitly allow traffic with origin/destination our neutron addresses:
Those are the addresses all outgoing traffic uses as source NAT, except traffic using floating IPs which also may be an overlapping setting anyway:
Mentioned in SAL (#wikimedia-cloud) [2020-03-11T12:50:56Z] <arturo> [codfw1dev] several tests creating/deleting address scopes (T244727 T247135 T246887 T245606)
Change 577575 abandoned by Ayounsi:
Add cloud-out4 firewall filter
Reason:
Experiment not successful