DNS CI is broken
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	jbond
	Jun 24 2022, 9:17 AM

Description

CI for DNS is failing wit the following error

11:03:20 E101|MULTIPLE_IPS_FOR_NAME: Found 3 IPs for name 'lvs4005.ulsfo.wmnet.', expected 1: 
11:03:20     netbox/ulsfo.wmnet:43 lvs4005.ulsfo.wmnet. A 10.128.0.15 
11:03:20     netbox/ulsfo.wmnet:44 lvs4005.ulsfo.wmnet. AAAA 2620:0:863:1:f6e9:d4ff:feba:f460 
11:03:20     netbox/ulsfo.wmnet:45 lvs4005.ulsfo.wmnet. AAAA 2620:0:863:101:10:128:0:15 
11:03:20 E101|MULTIPLE_IPS_FOR_NAME: Found 3 IPs for name 'lvs4006.ulsfo.wmnet.', expected 1: 
11:03:20     netbox/ulsfo.wmnet:47 lvs4006.ulsfo.wmnet. A 10.128.0.16 
11:03:20     netbox/ulsfo.wmnet:48 lvs4006.ulsfo.wmnet. AAAA 2620:0:863:1:f6e9:d4ff:feba:e550 
11:03:20     netbox/ulsfo.wmnet:49 lvs4006.ulsfo.wmnet. AAAA 2620:0:863:101:10:128:0:16 
11:03:20 E101|MULTIPLE_IPS_FOR_NAME: Found 3 IPs for name 'lvs4007.ulsfo.wmnet.', expected 1: 
11:03:20     netbox/ulsfo.wmnet:51 lvs4007.ulsfo.wmnet. A 10.128.0.17 
11:03:20     netbox/ulsfo.wmnet:52 lvs4007.ulsfo.wmnet. AAAA 2620:0:863:1:f6e9:d4ff:feba:eef0 
11:03:20     netbox/ulsfo.wmnet:53 lvs4007.ulsfo.wmnet. AAAA 2620:0:863:101:10:128:0:17 
11:03:20 Summary of violations:
11:03:20     E101|MULTIPLE_IPS_FOR_NAME: 3

This is related to the lvs4007 netbox entry that was update by @BCornwall on the 23/06/2020

Related Objects

Mentioned In: T271144: Some Traffic clusters apparently do not support IPv6
T311241: DNS/networking not working on Trusted Runners
T311133: Site: eqiad : 2 VMs requested for DSE Kubernetes Cluster control plane servers

Event Timeline

jbond triaged this task as High priority.Jun 24 2022, 9:17 AM

jbond created this task.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 24 2022, 9:17 AM

Maintenance_bot added a project: SRE.Jun 24 2022, 9:29 AM

Maintenance_bot added a project: Traffic.

Error message can be a bit criptic but if deployed to DNS this would have meant that:
lvs4005.ulsfo.wmnet (and similar) pointed to both AAAA 2620:0:863:1:f6e9:d4ff:feba:f460 and AAAA 2620:0:863:101:10:128:0:15 which is invalid in our infrastructure.

See the new dns_name I set for those IPs https://netbox.wikimedia.org/ipam/ip-addresses/?q=vl1201

As they're public IPs I set them in the wikimedia.org realm to avoid confusion as well.

Dzahn mentioned this in T311133: Site: eqiad : 2 VMs requested for DSE Kubernetes Cluster control plane servers.Jun 24 2022, 6:56 PM

Dzahn mentioned this in T311241: DNS/networking not working on Trusted Runners.Jun 27 2022, 9:18 PM

ayounsi mentioned this in T271144: Some Traffic clusters apparently do not support IPv6.Jul 7 2022, 7:05 AM

DNS CI is brokenClosed, ResolvedPublicActions

Description

Related Objects

Event Timeline

DNS CI is broken
Closed, ResolvedPublic
Actions