Page MenuHomePhabricator

Open swift port (28080) to the public internet
Closed, ResolvedPublic

Description

In theory this file should be public everywhere:

https://openstack.codfw1dev.wikimediacloud.org:28080/swift/v1/AUTH_testlabs/testcontainer/firstfile

It isn't, which has me thinking that we need a firewall or routing change for 28080.

Event Timeline

ah, nevermind, this is because no codfw1dev API is publicly visible.

Hmm, I think for swift to be useful we will need to expose it publicly so that for example web tools can embed data directly from Swift instead of proxying it. And for that (usage in browsers) I think we should be using the standard port 443. Unless the API and public file access are routed somehow differently?

Change 936657 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] codfw1dev: open radosgw API to the internet

https://gerrit.wikimedia.org/r/936657

ah, nevermind, this is because no codfw1dev API is publicly visible.

The patch https://gerrit.wikimedia.org/r/936657 should open it.

I'll let you decide if we want it merged or not.

Being codfw1dev, I'm a bit reluctant to have stuff exposed to the internet, so my recommendation would be no! and run tests and stuff from inside the WMF network.

aborrero moved this task from Backlog to Radar on the User-aborrero board.

Hmm, I think for swift to be useful we will need to expose it publicly so that for example web tools can embed data directly from Swift instead of proxying it.

Agreed for eqiad1, probably not for codfw1dev

And for that (usage in browsers) I think we should be using the standard port 443. Unless the API and public file access are routed somehow differently?

This I'm not sure about. The swift UI suggests urls like the above (https://openstack.codfw1dev.wikimediacloud.org:28080/swift/v1/AUTH_testlabs/testcontainer/firstfile) for accessing objects, and that seems to work ok with e.g. wget. We could provide some kind of middleware that reroutes from port 443 on e.g. swift.openstack.<whatever> but would that necessarily be better?

but would that necessarily be better?

I think yes. This service will be directly accessed by end users who might have weird corporate firewalls or similar restricting outbound traffic to non-standard ports.

Change 963325 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Move radosgw/swift API to port 443, the standard swift port

https://gerrit.wikimedia.org/r/963325

Change 963325 merged by Andrew Bogott:

[operations/puppet@production] Move radosgw/swift API to port 443, the standard swift port

https://gerrit.wikimedia.org/r/963325

Change 963330 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/dns@master] wikimediacloud: Add a dedicated CNAME for object storage

https://gerrit.wikimedia.org/r/963330

I've moved this service to port 443, which is open in eqiad1.

Looking on one of the cloudlb hosts in codfw it doesn't look like port 443 is open to the world:

cmooney@cloudlb2001-dev:~$ sudo iptables -L -v --line -n | grep "dpt:443"
1582     0     0 ACCEPT     tcp  --  *      *       10.128.0.0/24        0.0.0.0/0            tcp dpt:443
1583     0     0 ACCEPT     tcp  --  *      *       10.132.0.0/24        0.0.0.0/0            tcp dpt:443
1584     0     0 ACCEPT     tcp  --  *      *       10.136.0.0/24        0.0.0.0/0            tcp dpt:443
1585     0     0 ACCEPT     tcp  --  *      *       10.136.1.0/24        0.0.0.0/0            tcp dpt:443
1586     0     0 ACCEPT     tcp  --  *      *       10.192.0.0/22        0.0.0.0/0            tcp dpt:443
1587     0     0 ACCEPT     tcp  --  *      *       10.192.10.0/24       0.0.0.0/0            tcp dpt:443
1588     0     0 ACCEPT     tcp  --  *      *       10.192.11.0/24       0.0.0.0/0            tcp dpt:443
1589     0     0 ACCEPT     tcp  --  *      *       10.192.12.0/24       0.0.0.0/0            tcp dpt:443
1590     0     0 ACCEPT     tcp  --  *      *       10.192.13.0/24       0.0.0.0/0            tcp dpt:443
1591     0     0 ACCEPT     tcp  --  *      *       10.192.14.0/24       0.0.0.0/0            tcp dpt:443
1592     0     0 ACCEPT     tcp  --  *      *       10.192.15.0/24       0.0.0.0/0            tcp dpt:443
1593     0     0 ACCEPT     tcp  --  *      *       10.192.16.0/22       0.0.0.0/0            tcp dpt:443
1594     0     0 ACCEPT     tcp  --  *      *       10.192.20.0/24       0.0.0.0/0            tcp dpt:443
1595     0     0 ACCEPT     tcp  --  *      *       10.192.21.0/24       0.0.0.0/0            tcp dpt:443
1596     0     0 ACCEPT     tcp  --  *      *       10.192.22.0/24       0.0.0.0/0            tcp dpt:443
1597     0     0 ACCEPT     tcp  --  *      *       10.192.23.0/24       0.0.0.0/0            tcp dpt:443
1598     0     0 ACCEPT     tcp  --  *      *       10.192.32.0/22       0.0.0.0/0            tcp dpt:443
1599     0     0 ACCEPT     tcp  --  *      *       10.192.4.0/24        0.0.0.0/0            tcp dpt:443
1600     0     0 ACCEPT     tcp  --  *      *       10.192.48.0/22       0.0.0.0/0            tcp dpt:443
1601     0     0 ACCEPT     tcp  --  *      *       10.192.5.0/24        0.0.0.0/0            tcp dpt:443
1602     0     0 ACCEPT     tcp  --  *      *       10.192.6.0/24        0.0.0.0/0            tcp dpt:443
1603     0     0 ACCEPT     tcp  --  *      *       10.192.7.0/24        0.0.0.0/0            tcp dpt:443
1604     0     0 ACCEPT     tcp  --  *      *       10.192.72.0/24       0.0.0.0/0            tcp dpt:443
1605     0     0 ACCEPT     tcp  --  *      *       10.192.75.0/24       0.0.0.0/0            tcp dpt:443
1606     0     0 ACCEPT     tcp  --  *      *       10.192.76.0/24       0.0.0.0/0            tcp dpt:443
1607     0     0 ACCEPT     tcp  --  *      *       10.192.8.0/24        0.0.0.0/0            tcp dpt:443
1608     0     0 ACCEPT     tcp  --  *      *       10.192.9.0/24        0.0.0.0/0            tcp dpt:443
1609     0     0 ACCEPT     tcp  --  *      *       10.194.0.0/20        0.0.0.0/0            tcp dpt:443
1610     0     0 ACCEPT     tcp  --  *      *       10.194.128.0/18      0.0.0.0/0            tcp dpt:443
1611     0     0 ACCEPT     tcp  --  *      *       10.194.16.0/21       0.0.0.0/0            tcp dpt:443
1612     0     0 ACCEPT     tcp  --  *      *       10.194.61.0/24       0.0.0.0/0            tcp dpt:443
1613     0     0 ACCEPT     tcp  --  *      *       10.194.62.0/23       0.0.0.0/0            tcp dpt:443
1614     0     0 ACCEPT     tcp  --  *      *       10.2.1.0/24          0.0.0.0/0            tcp dpt:443
1615     0     0 ACCEPT     tcp  --  *      *       10.2.2.0/24          0.0.0.0/0            tcp dpt:443
1616     0     0 ACCEPT     tcp  --  *      *       10.2.3.0/24          0.0.0.0/0            tcp dpt:443
1617     0     0 ACCEPT     tcp  --  *      *       10.2.4.0/24          0.0.0.0/0            tcp dpt:443
1618     0     0 ACCEPT     tcp  --  *      *       10.2.5.0/24          0.0.0.0/0            tcp dpt:443
1619     0     0 ACCEPT     tcp  --  *      *       10.2.6.0/24          0.0.0.0/0            tcp dpt:443
1620     0     0 ACCEPT     tcp  --  *      *       10.64.0.0/22         0.0.0.0/0            tcp dpt:443
1621     0     0 ACCEPT     tcp  --  *      *       10.64.130.0/24       0.0.0.0/0            tcp dpt:443
1622     0     0 ACCEPT     tcp  --  *      *       10.64.131.0/24       0.0.0.0/0            tcp dpt:443
1623     0     0 ACCEPT     tcp  --  *      *       10.64.132.0/24       0.0.0.0/0            tcp dpt:443
1624     0     0 ACCEPT     tcp  --  *      *       10.64.134.0/24       0.0.0.0/0            tcp dpt:443
1625     0     0 ACCEPT     tcp  --  *      *       10.64.135.0/24       0.0.0.0/0            tcp dpt:443
1626     0     0 ACCEPT     tcp  --  *      *       10.64.136.0/24       0.0.0.0/0            tcp dpt:443
1627     0     0 ACCEPT     tcp  --  *      *       10.64.138.0/24       0.0.0.0/0            tcp dpt:443
1628     0     0 ACCEPT     tcp  --  *      *       10.64.139.0/24       0.0.0.0/0            tcp dpt:443
1629     0     0 ACCEPT     tcp  --  *      *       10.64.140.0/24       0.0.0.0/0            tcp dpt:443
1630     0     0 ACCEPT     tcp  --  *      *       10.64.142.0/24       0.0.0.0/0            tcp dpt:443
1631     0     0 ACCEPT     tcp  --  *      *       10.64.143.0/24       0.0.0.0/0            tcp dpt:443
1632     0     0 ACCEPT     tcp  --  *      *       10.64.144.0/24       0.0.0.0/0            tcp dpt:443
1633     0     0 ACCEPT     tcp  --  *      *       10.64.148.0/24       0.0.0.0/0            tcp dpt:443
1634     0     0 ACCEPT     tcp  --  *      *       10.64.149.0/24       0.0.0.0/0            tcp dpt:443
1635     0     0 ACCEPT     tcp  --  *      *       10.64.150.0/24       0.0.0.0/0            tcp dpt:443
1636     0     0 ACCEPT     tcp  --  *      *       10.64.151.0/24       0.0.0.0/0            tcp dpt:443
1637     0     0 ACCEPT     tcp  --  *      *       10.64.16.0/22        0.0.0.0/0            tcp dpt:443
1638     0     0 ACCEPT     tcp  --  *      *       10.64.20.0/24        0.0.0.0/0            tcp dpt:443
1639     0     0 ACCEPT     tcp  --  *      *       10.64.21.0/24        0.0.0.0/0            tcp dpt:443
1640     0     0 ACCEPT     tcp  --  *      *       10.64.32.0/22        0.0.0.0/0            tcp dpt:443
1641     0     0 ACCEPT     tcp  --  *      *       10.64.36.0/24        0.0.0.0/0            tcp dpt:443
1642     0     0 ACCEPT     tcp  --  *      *       10.64.37.0/24        0.0.0.0/0            tcp dpt:443
1643     0     0 ACCEPT     tcp  --  *      *       10.64.4.0/24         0.0.0.0/0            tcp dpt:443
1644     0     0 ACCEPT     tcp  --  *      *       10.64.48.0/22        0.0.0.0/0            tcp dpt:443
1645     0     0 ACCEPT     tcp  --  *      *       10.64.5.0/24         0.0.0.0/0            tcp dpt:443
1646     0     0 ACCEPT     tcp  --  *      *       10.64.52.0/24        0.0.0.0/0            tcp dpt:443
1647     0     0 ACCEPT     tcp  --  *      *       10.64.53.0/24        0.0.0.0/0            tcp dpt:443
1648     0     0 ACCEPT     tcp  --  *      *       10.64.72.0/24        0.0.0.0/0            tcp dpt:443
1649     0     0 ACCEPT     tcp  --  *      *       10.64.75.0/24        0.0.0.0/0            tcp dpt:443
1650     0     0 ACCEPT     tcp  --  *      *       10.64.76.0/24        0.0.0.0/0            tcp dpt:443
1651     0     0 ACCEPT     tcp  --  *      *       10.67.0.0/20         0.0.0.0/0            tcp dpt:443
1652     0     0 ACCEPT     tcp  --  *      *       10.67.128.0/18       0.0.0.0/0            tcp dpt:443
1653     0     0 ACCEPT     tcp  --  *      *       10.67.16.0/21        0.0.0.0/0            tcp dpt:443
1654     0     0 ACCEPT     tcp  --  *      *       10.67.24.0/21        0.0.0.0/0            tcp dpt:443
1655     0     0 ACCEPT     tcp  --  *      *       10.67.32.0/20        0.0.0.0/0            tcp dpt:443
1656     0     0 ACCEPT     tcp  --  *      *       10.67.64.0/20        0.0.0.0/0            tcp dpt:443
1657     0     0 ACCEPT     tcp  --  *      *       10.67.80.0/21        0.0.0.0/0            tcp dpt:443
1658     0     0 ACCEPT     tcp  --  *      *       10.80.0.0/24         0.0.0.0/0            tcp dpt:443
1659     0     0 ACCEPT     tcp  --  *      *       10.80.1.0/24         0.0.0.0/0            tcp dpt:443
1660     0     0 ACCEPT     tcp  --  *      *       103.102.166.0/28     0.0.0.0/0            tcp dpt:443
1661     0     0 ACCEPT     tcp  --  *      *       103.102.166.224/27   0.0.0.0/0            tcp dpt:443
1662     0     0 ACCEPT     tcp  --  *      *       172.16.0.0/21        0.0.0.0/0            tcp dpt:443
1663     0     0 ACCEPT     tcp  --  *      *       172.16.128.0/24      0.0.0.0/0            tcp dpt:443
1664     0     0 ACCEPT     tcp  --  *      *       172.20.1.0/24        0.0.0.0/0            tcp dpt:443
1665     0     0 ACCEPT     tcp  --  *      *       172.20.2.0/24        0.0.0.0/0            tcp dpt:443
1666     0     0 ACCEPT     tcp  --  *      *       172.20.255.0/24      0.0.0.0/0            tcp dpt:443
1667     0     0 ACCEPT     tcp  --  *      *       172.20.3.0/24        0.0.0.0/0            tcp dpt:443
1668     0     0 ACCEPT     tcp  --  *      *       172.20.4.0/24        0.0.0.0/0            tcp dpt:443
1669     0     0 ACCEPT     tcp  --  *      *       172.20.5.0/24        0.0.0.0/0            tcp dpt:443
1670     0     0 ACCEPT     tcp  --  *      *       185.15.56.0/25       0.0.0.0/0            tcp dpt:443
1671     0     0 ACCEPT     tcp  --  *      *       185.15.56.160/28     0.0.0.0/0            tcp dpt:443
1672     0     0 ACCEPT     tcp  --  *      *       185.15.57.0/29       0.0.0.0/0            tcp dpt:443
1673     0     0 ACCEPT     tcp  --  *      *       185.15.57.16/29      0.0.0.0/0            tcp dpt:443
1674     0     0 ACCEPT     tcp  --  *      *       185.15.57.24/29      0.0.0.0/0            tcp dpt:443
1675     0     0 ACCEPT     tcp  --  *      *       185.15.58.0/27       0.0.0.0/0            tcp dpt:443
1676     0     0 ACCEPT     tcp  --  *      *       185.15.58.224/27     0.0.0.0/0            tcp dpt:443
1677     0     0 ACCEPT     tcp  --  *      *       185.15.58.32/27      0.0.0.0/0            tcp dpt:443
1678     0     0 ACCEPT     tcp  --  *      *       185.15.59.0/27       0.0.0.0/0            tcp dpt:443
1679     0     0 ACCEPT     tcp  --  *      *       185.15.59.224/27     0.0.0.0/0            tcp dpt:443
1680     0     0 ACCEPT     tcp  --  *      *       185.15.59.32/27      0.0.0.0/0            tcp dpt:443
1681     0     0 ACCEPT     tcp  --  *      *       198.35.26.0/28       0.0.0.0/0            tcp dpt:443
1682     0     0 ACCEPT     tcp  --  *      *       198.35.26.96/27      0.0.0.0/0            tcp dpt:443
1683     0     0 ACCEPT     tcp  --  *      *       208.80.153.0/27      0.0.0.0/0            tcp dpt:443
1684     0     0 ACCEPT     tcp  --  *      *       208.80.153.224/27    0.0.0.0/0            tcp dpt:443
1685     0     0 ACCEPT     tcp  --  *      *       208.80.153.32/27     0.0.0.0/0            tcp dpt:443
1686     0     0 ACCEPT     tcp  --  *      *       208.80.153.64/27     0.0.0.0/0            tcp dpt:443
1687     0     0 ACCEPT     tcp  --  *      *       208.80.153.96/27     0.0.0.0/0            tcp dpt:443
1688     0     0 ACCEPT     tcp  --  *      *       208.80.154.0/26      0.0.0.0/0            tcp dpt:443
1689     0     0 ACCEPT     tcp  --  *      *       208.80.154.128/26    0.0.0.0/0            tcp dpt:443
1690     0     0 ACCEPT     tcp  --  *      *       208.80.154.224/27    0.0.0.0/0            tcp dpt:443
1691     0     0 ACCEPT     tcp  --  *      *       208.80.154.64/26     0.0.0.0/0            tcp dpt:443
1692     0     0 ACCEPT     tcp  --  *      *       208.80.155.96/27     0.0.0.0/0            tcp dpt:443

The CR routers don't block any traffic from public IPs like 185.15.57.24 (openstack.codfw1dev.wikimediacloud.org) routed in the cloud vrf, so I think the only place that it needs to be allowed is on the cloud hosts themselves. A curl from my home machine doesn't get a response, but the packets hit cloudlb2002-dev:

cathal@officepc:~$ curl -v https://openstack.codfw1dev.wikimediacloud.org/swift/v1/AUTH_testlabs/testcontainer/firstfile
*   Trying 185.15.57.24:443...
* connect to 185.15.57.24 port 443 failed: Connection timed out
* Failed to connect to openstack.codfw1dev.wikimediacloud.org port 443 after 129334 ms: Connection timed out
cmooney@cloudlb2002-dev:~$ sudo tcpdump -l -p -nn -i vlan2151 host 185.15.57.24 and tcp port 443
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vlan2151, link-type EN10MB (Ethernet), snapshot length 262144 bytes
17:33:41.875842 IP 176.61.34.2.56374 > 185.15.57.24.443: Flags [S], seq 1305605804, win 64240, options [mss 1460,sackOK,TS val 381362388 ecr 0,nop,wscale 7], length 0
17:33:42.904491 IP 176.61.34.2.56374 > 185.15.57.24.443: Flags [S], seq 1305605804, win 64240, options [mss 1460,sackOK,TS val 381363417 ecr 0,nop,wscale 7], length 0
17:33:44.921946 IP 176.61.34.2.56374 > 185.15.57.24.443: Flags [S], seq 1305605804, win 64240, options [mss 1460,sackOK,TS val 381365434 ecr 0,nop,wscale 7], length 0

Change 963330 merged by Majavah:

[operations/dns@master] wikimediacloud: Add a dedicated CNAME for object storage

https://gerrit.wikimedia.org/r/963330

Looking on one of the cloudlb hosts in codfw it doesn't look like port 443 is open to the world:

That's on purpose. There's an 'open_to_internet' flag in hiera that determines if a given service is firewalled or not; for codfw1dev we've been leaving the apis closed off. It's easy to switch if we need it later.

Change 936657 abandoned by Andrew Bogott:

[operations/puppet@production] codfw1dev: open radosgw API to the internet

Reason:

As a rule we don't have codfw1dev apis open to the public.

https://gerrit.wikimedia.org/r/936657