Repurposing this task to cover the use case to catch duplicates between manual records in the operation/dns repository and the automatically-generated DNS data coming from Netbox. More details in T155761#7893098.
It happened more than once in the last few months that the same IP was assigned to multiple hosts in the DNS repo.
We should add a check before merging to ensure that this cannot happen again.
As discussed in IRC, there doesn't seem to be any reason to have duplicate A entries in the forward files.
AFAICT nowadays zone_validator.py will fail on duplicate records. Ok to resolve this @Volans ?
@fgiunchedi yes and no, duplicates within the operations/dns repository are currently catched, but duplication within the automatically-generated data or between the manual and the generated data are not.
What we could do is to refactor a bit zone validator to inject into the zonefiles the netbox generated data for each INCLUDE before parsing the file. That should allow to catch all issues, but would also mean that some wrong data in Netbox might make CI fail on a totally valid dns patch.
What are your thoughts?
Thank you for the clarification @Volans, we're definitely in a better place nowadays so IMHO this task is done according to the original description.
What you mentioned sounds definitely worth doing, either as part of repurposing this task or a new one (both with more up to date tags I suspect, such as SRE-tools perhaps?). I'm fine either way!
I've a local patch that I'm testing to perform the validation of the whole dataset (manual + netbox). The preliminary results are below. I will have a look at the reported errors (that seems legit at first sight) and also at the warnings that might not be reported correctly anymore (some seems a bit too many).
-- Running zone_validator to check WMF rules
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 43
W002|MISSING_PTR_FOR_NAME_AND_IP: 29
W103|MISSING_MGMT_FOR_NAME: 66
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 141 Warnings, 0 Ignored violations, 0 Ignored lines -- Running zone_validator to check WMF rules
E001|GLOBAL_DUPLICATE: Global duplicate records found:
netbox/wikimedia.org-codfw:175 pfw3-codfw.wikimedia.org. A 208.80.153.197
wikimedia.org:296 pfw3-codfw.wikimedia.org. A 208.80.153.197
E001|GLOBAL_DUPLICATE: Global duplicate records found:
netbox/wikimedia.org-codfw:175 pfw3-codfw.wikimedia.org. A 208.80.153.197
wikimedia.org:296 pfw3-codfw.wikimedia.org. A 208.80.153.197
E001|GLOBAL_DUPLICATE: Global duplicate records found:
netbox/wikimedia.org-codfw:196 wiki-mail-codfw.wikimedia.org. A 208.80.153.46
netbox/wikimedia.org-codfw:197 wiki-mail-codfw.wikimedia.org. AAAA 2620:0:860:2:208:80:153:46
wikimedia.org:463 wiki-mail-codfw.wikimedia.org. A 208.80.153.46
wikimedia.org:464 wiki-mail-codfw.wikimedia.org. AAAA 2620:0:860:2:208:80:153:46
E001|GLOBAL_DUPLICATE: Global duplicate records found:
netbox/wikimedia.org-codfw:196 wiki-mail-codfw.wikimedia.org. A 208.80.153.46
netbox/wikimedia.org-codfw:197 wiki-mail-codfw.wikimedia.org. AAAA 2620:0:860:2:208:80:153:46
wikimedia.org:463 wiki-mail-codfw.wikimedia.org. A 208.80.153.46
wikimedia.org:464 wiki-mail-codfw.wikimedia.org. AAAA 2620:0:860:2:208:80:153:46
E001|GLOBAL_DUPLICATE: Global duplicate records found:
netbox/wikimedia.org-codfw:196 wiki-mail-codfw.wikimedia.org. A 208.80.153.46
netbox/wikimedia.org-codfw:197 wiki-mail-codfw.wikimedia.org. AAAA 2620:0:860:2:208:80:153:46
wikimedia.org:463 wiki-mail-codfw.wikimedia.org. A 208.80.153.46
wikimedia.org:464 wiki-mail-codfw.wikimedia.org. AAAA 2620:0:860:2:208:80:153:46
E001|GLOBAL_DUPLICATE: Global duplicate records found:
netbox/wikimedia.org-codfw:196 wiki-mail-codfw.wikimedia.org. A 208.80.153.46
netbox/wikimedia.org-codfw:197 wiki-mail-codfw.wikimedia.org. AAAA 2620:0:860:2:208:80:153:46
wikimedia.org:463 wiki-mail-codfw.wikimedia.org. A 208.80.153.46
wikimedia.org:464 wiki-mail-codfw.wikimedia.org. AAAA 2620:0:860:2:208:80:153:46
E001|GLOBAL_DUPLICATE: Global duplicate records found:
netbox/32-27.153.80.208.in-addr.arpa:13 47.153.80.208.in-addr.arpa. PTR ns-recursor0.openstack.codfw1dev.wikimediacloud.org.
153.80.208.in-addr.arpa:23 47.153.80.208.in-addr.arpa. PTR ns-recursor0.openstack.codfw1dev.wikimediacloud.org. ; additional IP for cloudservices2004-dev
E001|GLOBAL_DUPLICATE: Global duplicate records found:
netbox/32-27.153.80.208.in-addr.arpa:13 47.153.80.208.in-addr.arpa. PTR ns-recursor0.openstack.codfw1dev.wikimediacloud.org.
153.80.208.in-addr.arpa:23 47.153.80.208.in-addr.arpa. PTR ns-recursor0.openstack.codfw1dev.wikimediacloud.org. ; additional IP for cloudservices2004-dev
E001|GLOBAL_DUPLICATE: Global duplicate records found:
netbox/184-29.153.80.208.in-addr.arpa:6 190.153.80.208.in-addr.arpa. PTR wan.cloudgw.codfw1dev.wikimediacloud.org.
153.80.208.in-addr.arpa:46 190.153.80.208.in-addr.arpa. PTR wan.cloudgw.codfw1dev.wikimediacloud.org.
E001|GLOBAL_DUPLICATE: Global duplicate records found:
netbox/184-29.153.80.208.in-addr.arpa:6 190.153.80.208.in-addr.arpa. PTR wan.cloudgw.codfw1dev.wikimediacloud.org.
153.80.208.in-addr.arpa:46 190.153.80.208.in-addr.arpa. PTR wan.cloudgw.codfw1dev.wikimediacloud.org.
E103|TOO_MANY_NAMES: Found 2 name(s) for PTR '190.153.80.208.in-addr.arpa.', expected 1:
netbox/184-29.153.80.208.in-addr.arpa:6 190.153.80.208.in-addr.arpa. PTR wan.cloudgw.codfw1dev.wikimediacloud.org.
153.80.208.in-addr.arpa:46 190.153.80.208.in-addr.arpa. PTR wan.cloudgw.codfw1dev.wikimediacloud.org.
E103|TOO_MANY_NAMES: Found 2 name(s) for PTR '47.153.80.208.in-addr.arpa.', expected 1:
netbox/32-27.153.80.208.in-addr.arpa:13 47.153.80.208.in-addr.arpa. PTR ns-recursor0.openstack.codfw1dev.wikimediacloud.org.
153.80.208.in-addr.arpa:23 47.153.80.208.in-addr.arpa. PTR ns-recursor0.openstack.codfw1dev.wikimediacloud.org. ; additional IP for cloudservices2004-dev
E103|TOO_MANY_NAMES: Found 2 name(s) for PTR '50.153.80.208.in-addr.arpa.', expected 1:
netbox/32-27.153.80.208.in-addr.arpa:16 50.153.80.208.in-addr.arpa. PTR ns-recursor0.openstack.codfw1dev.wikimediacloud.org.
153.80.208.in-addr.arpa:24 50.153.80.208.in-addr.arpa. PTR ns-recursor1.openstack.codfw1dev.wikimediacloud.org. ; additional IP for cloudservices2005-dev
E101|MULTIPLE_IPS_FOR_NAME: Found 3 IPs for name 'ge-0-0-4-401.mr1-eqsin.wikimedia.org.', expected 1:
netbox/wikimedia.org-eqsin:61 ge-0-0-4-401.mr1-eqsin.wikimedia.org. A 103.102.166.133
netbox/wikimedia.org-eqsin:62 ge-0-0-4-401.mr1-eqsin.wikimedia.org. AAAA 2001:df2:e500:fe03::2
netbox/wikimedia.org-eqsin:63 ge-0-0-4-401.mr1-eqsin.wikimedia.org. AAAA 2001:df2:e500:fe04::2
E101|MULTIPLE_IPS_FOR_NAME: Found 4 IPs for name 'wiki-mail-codfw.wikimedia.org.', expected 1:
netbox/wikimedia.org-codfw:196 wiki-mail-codfw.wikimedia.org. A 208.80.153.46
netbox/wikimedia.org-codfw:197 wiki-mail-codfw.wikimedia.org. AAAA 2620:0:860:2:208:80:153:46
wikimedia.org:463 wiki-mail-codfw.wikimedia.org. A 208.80.153.46
wikimedia.org:464 wiki-mail-codfw.wikimedia.org. AAAA 2620:0:860:2:208:80:153:46
E003|MISSING_OR_WRONG_PTR_FOR_NAME_AND_IP: Missing PTR '1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.4.0.e.f.0.0.6.0.0.8.c.e.2.0.a.2.ip6.arpa.' for name 'xe-0-1-1-16.cr2-drmrs.wikimedia.org.' and IP '2a02:ec80:600:fe04::1', PTRs are:
146.58.15.185.in-addr.arpa.
E003|MISSING_OR_WRONG_PTR_FOR_NAME_AND_IP: Missing PTR '2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.4.0.e.f.0.0.6.0.0.8.c.e.2.0.a.2.ip6.arpa.' for name 'xe-4-2-2-16.cr1-eqiad.wikimedia.org.' and IP '2a02:ec80:600:fe04::2', PTRs are:
147.58.15.185.in-addr.arpa.
Summary of violations:
E001|GLOBAL_DUPLICATE: 10
E003|MISSING_OR_WRONG_PTR_FOR_NAME_AND_IP: 2
E101|MULTIPLE_IPS_FOR_NAME: 2
E103|TOO_MANY_NAMES: 3
W001|MISSING_IP_FOR_NAME_AND_PTR: 37
W002|MISSING_PTR_FOR_NAME_AND_IP: 63
W101|MISSING_ASSET_TAG: 514
W102|MISSING_DUAL_STACK_FOR_NAME: 1
W103|MISSING_MGMT_FOR_NAME: 971
W104|TOO_FEW_MGMT_NAMES: 362
W105|TOO_MANY_PUBLIC_NAMES: 18
RESULT: 17 Errors, 1966 Warnings, 0 Ignored violations, 0 Ignored linesChange 793001 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] drmrs: add missing Netbox include for PTRs
Change 793002 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] pfw3-codfw: remove manual record managed by Netbox
Change 793003 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] cloud codfw1dev: remove records managed by Netbox
Change 793004 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] iwiki-mail-codfw: remove records managed by Netbox
Change 793001 merged by Volans:
[operations/dns@master] drmrs: add missing Netbox include for PTRs
Change 793002 merged by Volans:
[operations/dns@master] pfw3-codfw: remove manual record managed by Netbox
Change 793003 merged by Volans:
[operations/dns@master] cloud codfw1dev: remove records managed by Netbox
Change 793004 merged by Volans:
[operations/dns@master] wiki-mail-codfw: remove records managed by Netbox
Change 793047 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] Remove unused PTRs from old experiment
Change 793047 merged by Volans:
[operations/dns@master] Remove unused PTRs from old experiment
Change 793050 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] cloud codfw1dev: remove records managed by Netbox
Change 793056 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] tox.ini: remove older Python versions, add 3.9
Change 793057 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] zone_validator: include Netbox data in the check
Change 793050 merged by Volans:
[operations/dns@master] cloud codfw1dev: fix recursor records
Change 793467 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] zone_validator: fix asset tag matching
Change 793468 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] zone_validator: add new zonefiles
Change 793469 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] zone_validator: improve output of reported issues
Change 793470 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] zone_validator: add support for @ records
Change 793471 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] zone_validator: fix inline ignore errors logic
Change 793472 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] zone_validator: convert format() and + to f-string
Change 793473 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] zone_validator: simplify ignore of multiple issues
Change 793484 had a related patch set uploaded (by Volans; author: Volans):
[operations/software/netbox-extras@master] dns: add a comment for skipped PTR
Change 793724 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] wikimedia-dns: add zone validator ignore comments
Change 793727 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] Non-WMF IPs: add zone validator ignore comments
Change 793728 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] Duplicate IPs by design: add zone validator ignore
Change 793729 had a related patch set uploaded (by Volans; author: Volans):
[operations/dns@master] wikitech-static-iad: remove obsolete records
Change 793724 merged by Volans:
[operations/dns@master] wikimedia-dns: add zone validator ignore comments
Change 793056 merged by Volans:
[operations/dns@master] tox.ini: remove older Python versions, add 3.9
Change 793467 merged by Volans:
[operations/dns@master] zone_validator: fix asset tag matching
Change 793468 merged by Volans:
[operations/dns@master] zone_validator: add new zonefiles
Change 793469 merged by Volans:
[operations/dns@master] zone_validator: improve output of reported issues
Change 793470 merged by Volans:
[operations/dns@master] zone_validator: add support for @ records
Change 793471 merged by Volans:
[operations/dns@master] zone_validator: fix inline ignore errors logic
Change 793057 merged by Volans:
[operations/dns@master] zone_validator: include Netbox data in the check
Change 793472 merged by Volans:
[operations/dns@master] zone_validator: convert format() and + to f-string
Change 793473 merged by Volans:
[operations/dns@master] zone_validator: simplify ignore of multiple issues
Change 793727 merged by Volans:
[operations/dns@master] Non-WMF IPs: add zone validator ignore comments
Change 793484 merged by jenkins-bot:
[operations/software/netbox-extras@master] dns: add a comment for skipped PTR
Change 793729 merged by Volans:
[operations/dns@master] wikitech-static-iad: remove obsolete records
I think that the only patch left to be merged is https://gerrit.wikimedia.org/r/c/operations/dns/+/793728, pending on Traffic for a review/approval.
@BBlack / @Vgutierrez is https://gerrit.wikimedia.org/r/c/operations/dns/+/793728 something that you're amenable to including?
Change #793728 merged by Ssingh:
[operations/dns@master] Duplicate names by design: add zone validator ignore
I believe this has been solved with the latest patch merge. If this is in error, please re-open!