Page MenuHomePhabricator
Create Task
Maniphest T306809

Enforce Netbox domain names without period termination
Closed, ResolvedPublic
Actions

Description

When adding names to prefixes such as on https://netbox.wikimedia.org/ipam/prefixes/34/ip-addresses/. if a user enters a fully qualified *and terminated* domain i.e. the domain ends with a period, into the hostname field then sre.dns.netbox generates reverse zones with a double period.

See the following diff after removing a period from some offending names

diff --git a/32-27.153.80.208.in-addr.arpa b/32-27.153.80.208.in-addr.arpa                                          
index 848c67c..c50f1c1 100644                                                                                       
--- a/32-27.153.80.208.in-addr.arpa                                                                                 
+++ b/32-27.153.80.208.in-addr.arpa                                                                                 
@@ -10,10 +10,10 @@                                                                                                 
 44  1H IN PTR cloudservices2005-dev.wikimedia.org.                                                                 
 45  1H IN PTR mx2001.wikimedia.org.                                                                                
 46  1H IN PTR wiki-mail-codfw.wikimedia.org.                                                                       
-47  1H IN PTR ns-recursor0.openstack.codfw1dev.wikimediacloud.org..                                                
+47  1H IN PTR ns-recursor0.openstack.codfw1dev.wikimediacloud.org.                                                 
 48  1H IN PTR ldap-corp2001.wikimedia.org.                                                                         
 49  1H IN PTR serpens.wikimedia.org.                                                                               
-50  1H IN PTR ns-recursor0.openstack.codfw1dev.wikimediacloud.org..                                                
+50  1H IN PTR ns-recursor0.openstack.codfw1dev.wikimediacloud.org.                                                 
 51  1H IN PTR install2003.wikimedia.org.                                                                           
 54  1H IN PTR bast2002.wikimedia.org.                                                                              
 55  1H IN PTR vl2002-enp59s0f1d1.lvs2007.codfw.wmnet.                                                              
diff --git a/org.-codfw b/org.-codfw                                                                                
index 1f1a321..e8914e7 100644                                                                                       
--- a/org.-codfw                                                                                                    
+++ b/org.-codfw                                                                                                    
@@ -1,4 +1,2 @@                                                                                                     
-ns-recursor0.openstack.codfw1dev.wikimediacloud 1H IN A 208.80.153.47                                              
-ns-recursor0.openstack.codfw1dev.wikimediacloud 1H IN A 208.80.153.50                                              
 ns-recursor0.openstack.codfw1dev.wikimediacloud 1H IN AAAA 2620:0:860:3:208:80:153:47                              
 ns-recursor1.openstack.codfw1dev.wikimediacloud 1H IN AAAA 2620:0:860:3:208:80:153:50                              
diff --git a/wikimediacloud.org-codfw b/wikimediacloud.org-codfw                                                    
index 710ae1f..67a0372 100644                                                                                       
--- a/wikimediacloud.org-codfw                                                                                      
+++ b/wikimediacloud.org-codfw                                                                                      
@@ -3,3 +3,5 @@ wan.cloudgw.codfw1dev                    1H IN A 208.80.153.190                                     
 cloudgw2001-dev.codfw1dev                1H IN A 208.80.153.188                                                    
 cloudgw2002-dev.codfw1dev                1H IN A 208.80.153.189                                                    
 cloudinstances2b-gw.openstack.codfw1dev  1H IN A 185.15.57.10                                                      
+ns-recursor0.openstack.codfw1dev         1H IN A 208.80.153.47                                                     
+ns-recursor0.openstack.codfw1dev         1H IN A 208.80.153.50

Beginning with version 3.0, Netbox allows custom data validation which would allow us to enforce consistency in formatting *without* the terminating period (due to the issue mentioned above).

Related Objects
Search...

Event Timeline

jbond created this task.Apr 25 2022, 3:30 PM
jbond triaged this task as Medium priority.
Restricted Application added a project: Infrastructure-Foundations. · View Herald TranscriptApr 25 2022, 3:30 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Maintenance_bot added a project: Traffic.Apr 25 2022, 3:30 PM
Maintenance_bot added a project: SRE.Apr 25 2022, 4:29 PM
Volans subscribed.Apr 26 2022, 8:49 AM

The DNS Name field in Netbox is an FQDN, the same Netbox UI help message for the field is: Hostname or FQDN (not case-sensitive)
For this reason I think that this task should be closed as invalid, and instead we should move towards having consistent data in Netbox.
The DNS Name field is used in multiple places in different automation, from Homer to various Netbox scripts and is always considered to be an FQDN.

jbond added a comment.Apr 26 2022, 9:13 AM

Im not sure i understand this response. The value entered which caused an error was ns-recursor0.openstack.codfw1dev.wikimediacloud.org. instead of ns-recursor0.openstack.codfw1dev.wikimediacloud.org both are valid FQDN and strictly speaking the one with the terminating period is the more correct form.

Volans added a comment.Apr 26 2022, 9:33 AM

Sure, but they could cause various unwanted issues in different contexes, like not matching the fingerprint in the known hosts file for SSH connections:

cumin1001 $ SSH_AUTH_SOCK=/run/keyholder/proxy.sock ssh root@sretest1001.eqiad.wmnet.
The authenticity of host 'sretest1001.eqiad.wmnet. (2620:0:861:107:10:64:48:138)' can't be established.
[...SNIP...]

That's why I would rather prefer to keep the data consistent in Netbox without the ending period, so that the behaviour is consistent everywhere. For the DNS -specific bits the period is automatically added always.
Thoughts?

jbond changed the task status from Open to Stalled.Apr 26 2022, 9:58 AM

As per an offline conversation with @Volans. newer versions of netbox allow us to preform custom data validations as such i'm going to set this ticket to stalled until we upgrade netbox to at least version 3.0

ayounsi added a subtask: T296452: Upgrade Netbox to 3.2.May 2 2022, 1:42 PM
ayounsi added a subtask: T310590: Netbox: use Custom Model Validation.Jun 24 2022, 7:26 AM
ayounsi changed the task status from Stalled to Open.Jun 24 2022, 7:45 AM
ayounsi closed subtask T296452: Upgrade Netbox to 3.2 as Resolved.Jul 19 2022, 12:56 PM
BCornwall moved this task from Backlog to Scheduled incidental work on the Traffic board.Feb 23 2023, 12:04 AM
BCornwall moved this task from Scheduled incidental work to Backlog on the Traffic board.Mar 30 2023, 8:43 PM
BCornwall subscribed.Mar 30 2023, 8:47 PM

FYI, we're on version 3 now!

@jbond, it's been a while since you had the conversation with @Volans but I'd like some clarification: Is the validation going to be used for ensuring that there *isn't* a terminating period so that the data is kept consistent?

BCornwall removed a project: SRE.Mar 30 2023, 8:48 PM
Volans added a comment.Mar 30 2023, 9:33 PM

Correct, and we've already the first validators in netbox-next that will be released to prod shortly so this can be surely one of the validators we could add soon.

BCornwall renamed this task from sre.dns.netbox cookbook dosn't support period terminated domains to Enforce Netbox domain names without period termination.Mar 31 2023, 3:26 PM
BCornwall updated the task description. (Show Details)
BCornwall removed a project: Traffic.

Updated the task description to accurately reflect the work that needs doing. I'm also going to remove the Traffic tag since it seems more relevant to IF - but if you feel otherwise, please re-add it!

ayounsi closed this task as Resolved.May 9 2023, 9:50 AM
ayounsi claimed this task.
ayounsi subscribed.

Done using Netbox validators.

ayounsi closed subtask T310590: Netbox: use Custom Model Validation as Resolved.Aug 19 2024, 11:29 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits