GitLab hosts use two public IPv4 addresses per host. With two hosts in codfw and two in eqiad 8 IPv4 addresses would be needed. Two of that hosts are still in setup and the old VMs gitlab1001 and gitlab2001 will be decommissioned soon, which will release some IPs. But long term that is not a scalable solution. During setup of new hosts in T307142 some discussions happened with Infrastructure Foundations about the usage of IPv4 addresses too.
This task is to track and discuss measures to reduce the number of used IPv4 addresses. There are two IPv4 addresses configured per GitLab host currently:
The primary interface is used to connect to the host over SSH for management purpose. This interface is configured with a public IP.
- Make the primary interface private (for example by moving gitlab1001.wikimedia.org to gitlab1001.eqiad.wmnet). SSH access is still possible with the standard bastion/jumphost configuration. That would halve IPv4 usage.
The second interface is used to serve http/https traffic for gitlab.wikimedia.org. Furthermore a second SSH daemon is listening on that address as well to separate it from the management SSH daemon. This services are used directly by end-users and need some kind of public endpoint.
- We can think about load balancing http/https and ssh traffic for the GitLab hosts and use a private second address. Some research is needed if existing loadbalancing infrastructure can handle SSH as well.
- GitLab replicas (non-production instances) could drop the public address while they are replicas. However this would significantly decrease the use of that replicas for tests and failovers.
I would like to experiment with private primary interfaces on the replicas. If that works fine, we can migrate production GitLab to a private primary interface as well.