Change Details

At the moment, Labs instances that want to be able to connect to "dewiki.labsdb" and end up at the DB replica server hosting the German Wikipedia, have to copy /etc/hosts from a Tools instance and set up iptables according to /data/project/.system/iptables.conf. I tried to puppetize this in Gerrit change #107010, but it turned out that using ferm (base::firewall) for iptables has severe side effects (like locking myself out from my test instance :-)). There is however an alternative that is rather easy to set up and maintain: Move the aliases to DNS, and the NAT rules to the DB servers itself. This way we only have to test (and worry) about three hosts and not dozens. This would include: - allocating seven IP addresses, - routing them to the LabsDB servers, - setting up a DNS zone labsdb with the aliases pointing at the IP addresses, - setting up firewall and NAT on the LabsDB servers. Thus, new instances in Tools and other Labs projects would have instant access (minus credentials) to the replica servers. Existing instances would not be affected. -------------------------- **Version**: unspecified **Severity**: enhancement **See Also**: https://bugzilla.wikimedia.org/show_bug.cgi?id=65308 https://bugzilla.wikimedia.org/show_bug.cgi?id=53181

At the moment, Labs instances that want to be able to connect to "dewiki.labsdb" and end up at the DB replica server hosting the German Wikipedia, have to copy /etc/hosts from a Tools instance. This system (per-database hostname) comes from the toolserver setup and is in active use by (most) tools that need to connect to project databases - in particular, tools which can operate on more than one project use the project name as a 'selector' in this way. The ideal solution would be to do away with this system entirely, and have tools resolve the proper database dynamically at runtime according to the information made available in every replica in the meta_p.wiki table. This, however, requires altering the code that runs those tools and possibly restructuring it according to the new scheme, something which is difficult to demand of every maintainer (not all of whom are active regularily). A decent intermediate solution is to have those aliases served properly by DNS as part of a subdomain, where only one copy of the data exists and needs to be maintained (and, since it's in git, can be maintained automatically if wanted). The current changeset proposes allocating labs.$site.wmnet for that purpose, and places the aliases in the zone file accordingly. resolv.conf on labs instances has already been taken by puppet to include 'labs.$site.wmnet' in the search paths, and set ndots to 2 meaning that the same hostnames will resolve through DNS the same way they did with /etc/hosts Possible improvements include delegating the subdomain to a labs server.

At the moment, Labs instances that want to be able to connect to "dewiki.labsdb" and end up at the DB replica server hosting the German Wikipedia, have to copy /etc/hosts from a Tools instance and set up iptables according to /data/project/.system/iptables.conf. This system (per-database hostname) comes from the toolserver setup and is in active use by (most) tools that need to connect to project databases - in particular, tools which can operate on more than one project use the project name as a 'selector' in this way. I tried to puppetize this in Gerrit change #107010The ideal solution would be to do away with this system entirely, and have tools resolve the proper database dynamically at runtime according to the information made available in every replica in the meta_p.wiki table. This, however, requires altering the code that runs those tools and possibly restructuring it according to the new scheme, but it turned out that using ferm (base::firewall) for iptables has severe side effects (like locking myself out from my test instance :-)something which is difficult to demand of every maintainer (not all of whom are active regularily). There is however an alternative that is rather easy to set up and maintain: Move the aliases to DNSA decent intermediate solution is to have those aliases served properly by DNS as part of a subdomain, where only one copy of the data exists and needs to be maintained (and, and the NAT rules to the DB servers itself. This way we only have to test (and worry) about three hosts and not dozenssince it's in git, can be maintained automatically if wanted). This would include:e current changeset proposes allocating labs.$site.wmnet for that purpose, and places the aliases in the zone file accordingly. resolv.conf on labs instances has already been taken by puppet to include 'labs.$site.wmnet' in the search paths, and set ndots to 2 meaning that the same hostnames will resolve through DNS the same way they did with /etc/hosts - allocPossible improvements include delegating seven IP addresses,the subdomain to a labs server. - routing them to the LabsDB servers, - setting up a DNS zone labsdb with the aliases pointing at the IP addresses, - setting up firewall and NAT on the LabsDB servers. Thus, new instances in Tools and other Labs projects would have instant access (minus credentials) to the replica servers. Existing instances would not be affected. -------------------------- **Version**: unspecified **Severity**: enhancement **See Also**: https://bugzilla.wikimedia.org/show_bug.cgi?id=65308 https://bugzilla.wikimedia.org/show_bug.cgi?id=53181