Page MenuHomePhabricator
Create Task
Maniphest T338938

codfw1dev: finish auth DNS server transition
Closed, InvalidPublic
Actions

Description

As part of T307357: Move cloud vps ns-recursor IPs to host/row-independent addressing we introduced new IP addresses for both the resolver and the auth DNS servers.

See https://gerrit.wikimedia.org/r/c/operations/dns/+/928620

However, the transition is not completed:

Until then, we may see some clients who think ns0/ns1 are still auth servers:

user@laptop:~$ dig @ns0.wikimedia.org NS codfw1dev.wikimedia.cloud
[..]
codfw1dev.wikimedia.cloud. 86400 IN	NS	ns.openstack.codfw1dev.wikimediacloud.org.

However:

user@laptop:~$ dig @ns.openstack.codfw1dev.wikimediacloud.org SOA codfw1dev.wikimedia.cloud +short
ns0.openstack.codfw1dev.wikimediacloud.org. root.wmflabs.org. 1686644138 3598 600 86400 3600
user@laptop:~$ dig @ns.openstack.codfw1dev.wikimediacloud.org NS codfw1dev.wikimedia.cloud +short
ns0.openstack.codfw1dev.wikimediacloud.org.
ns1.openstack.codfw1dev.wikimediacloud.org.
ns1.openstack.eqiad1.wikimediacloud.org.
ns0.openstack.eqiad1.wikimediacloud.org.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
openstack: codfw1dev: designate: listen-on only the new addressoperations/puppetproduction+0 -2
openstack: codfw1dev: designate: fix pdns query-local-adressoperations/puppetproduction+13 -10
pdns_server: make default-soa-content configurableoperations/puppetproduction+9 -21
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 Resolved• aborreroT296411 cloud: decide on general idea for having cloud-dedicated hardware provide service in the cloud realm & the internet
 Resolved• aborreroT297596 have cloud hardware servers in the cloud realm using a dedicated LB layer
 Resolved• aborreroT324992 cloudlb: create PoC on codfw
 Resolved• aborreroT327908 rename cloudgw2001-dev into cloudlb2001-dev
 ResolvedayounsiT327930 Is Vlan 2122 cloud-support1-b-codfw required?
 ResolvedcmooneyT327919 Configure cloudsw1-b1-codfw and migrate cloud hosts in codfw B1 to it
 ResolvedPapaulT331470 Run 2x1G links from asw-b1-codfw to cloudsw1-b1-codfw
Restricted Task
 ResolvedcmooneyT333316 Homer unable to commit config to cloudsw1-b1-codfw (QFX5120 21.4R3.16)
 ResolvedcmooneyT336368 cloudgw: review security policy for edge network
 Resolved• aborreroT332153 cloudlb PoC: prepare backends
 Resolved• aborreroT336963 cloudcontrol2001-dev can't reach cloud-vps public IPs
 Resolved• aborreroT335759 cloud-private subnet: introduce new domain
 ResolvedtaaviT336566 cloud: failures resolving some `wikimedia.cloud` domains
 OpencmooneyT346428 Netbox: Add support for our complex host network setups in provision script
 Resolved• aborreroT346410 need private IPs for cloudvirt200[4-6]-dev.codfw.wmnet
 OpencmooneyT347375 Netbox device location information not available on the first Puppet run of a device
 Resolved• aborreroT335760 Modify Bird module to allow source IP to be passed to template
 Resolved• aborreroT336071 cloudlb: figure out routing
 Resolved• aborreroT337758 puppet: interface::route resource does not persists settings
 Resolved• aborreroT338936 cloudlb: figure out plans for eqiad1
 Resolved• aborreroT338937 cloudlb: review swift/radosgw status
ResolvedAndrewT341380 Open swift port (28080) to the public internet
 ResolvedAndrewT341484 Horizon object store UI doesn't allow uploading of files
 ResolvedAndrewT341640 Move Horizon deploy to a Docker container
 ResolvedAndrewT341509 radosgw+keystone chokes on projects with '-' in their id
 ResolvedAndrewT343158 Support OpenStack projects where project name != project id
 ResolvedAndrewT366679 openstack-browser support for projects where id != name
 ResolvedAndrewT347927 Better swift error messages for projects with invalid chars
 ResolvedAndrewT366301 Document and enforce restrictions on openstack project names
 Resolved• aborreroT340536 cloud-private: figure out, implement and test cross-DC traffic
 OpenNoneT317177 [tracking] Don't keep on the public vlans hosts that don't require it
 ResolvedNoneT340446 Cloud VPS Designate setup improvements
 Resolved• aborreroT307357 Move cloud vps ns-recursor IPs to host/row-independent addressing
 ResolvedcmooneyT336587 cloudservices[2004/2005]-dev & cloudweb2002-dev: connect them to cloudsw so they can have cloud-private vlan
 Invalid• aborreroT338938 codfw1dev: finish auth DNS server transition

Event Timeline

• aborrero created this task.Jun 13 2023, 10:45 AM
• aborrero added a subtask: T338929: openstack designate: introduce some as-code mechanism to instrument DNS data.
• aborrero added a comment.Jun 13 2023, 12:24 PM

Just found this:

aborrero@cloudservices2005-dev:~ $ sudo grep -iR ns0 /etc/powerdns/
/etc/powerdns/pdns.conf:default-soa-content=ns0.openstack.eqiad1.wikimediacloud.org
aborrero@cloudservices2005-dev:~ $ sudo grep -iR ns0 /etc/designate/
/etc/designate/pools.yaml:  - hostname: ns0.openstack.codfw1dev.wikimediacloud.org.

This may be coded in puppet after all!

gerritbot added a comment.Jun 13 2023, 12:55 PM

Change 929697 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] pdns_server: make default-soa-content configurable

https://gerrit.wikimedia.org/r/929697

gerritbot added a project: Patch-For-Review.Jun 13 2023, 12:55 PM
gerritbot added a comment.Jun 13 2023, 2:25 PM

Change 929697 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] pdns_server: make default-soa-content configurable

https://gerrit.wikimedia.org/r/929697

Maintenance_bot removed a project: Patch-For-Review.Jun 13 2023, 2:29 PM
gerritbot added a comment.Jun 13 2023, 3:55 PM

Change 929739 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] openstack: codfw1dev: designate: fix pdns query-local-adress

https://gerrit.wikimedia.org/r/929739

gerritbot added a project: Patch-For-Review.Jun 13 2023, 3:55 PM

Change 929740 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[operations/puppet@production] openstack: codfw1dev: designate: listen-on only the new address

https://gerrit.wikimedia.org/r/929740

gerritbot added a comment.Jun 13 2023, 4:08 PM

Change 929739 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] openstack: codfw1dev: designate: fix pdns query-local-adress

https://gerrit.wikimedia.org/r/929739

• aborrero moved this task from Backlog to Doing on the User-aborrero board.Jun 13 2023, 4:32 PM
• aborrero moved this task from Doing to Next on the User-aborrero board.
• aborrero closed this task as Invalid.Jun 20 2023, 3:53 PM

The original plan was reworking mid-flight and we went back to having ns0/ns1 addresses for the hosts. This is because each DNS server is both Auth and Recursor, and each recursor instance needs a publicly reachable IPv4 address in order to, well, recurse.

So we re-introduced the per-host address.

• aborrero removed a subtask: T338929: openstack designate: introduce some as-code mechanism to instrument DNS data.Jun 26 2023, 2:49 PM
gerritbot added a comment.Dec 7 2023, 1:55 PM

Change 929740 abandoned by Majavah:

[operations/puppet@production] openstack: codfw1dev: designate: listen-on only the new address

Reason:

don't think this is still needed

https://gerrit.wikimedia.org/r/929740

Maintenance_bot removed a project: Patch-For-Review.Dec 7 2023, 2:10 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits