Page MenuHomePhabricator

LDAP contains two extra incorrect host entries with aRecord=10.68.17.118, one with aRecord=10.68.22.5, and one with aRecord=10.68.16.120
Closed, ResolvedPublic

Description

krenair@silver:~$ OS_TENANT_NAME=admin nova list --all-tenants | grep 10.68.17.118
| 508d0ca1-fe5e-4b8c-9034-cd5ffcb774e5 | integration-slave-trusty-1006        | integration                 | ACTIVE  | -          | Running     | public=10.68.17.118                                 |
krenair@silver:~$

And yet:

krenair@bastion-01:~$ ldapsearch -x aRecord=10.68.17.118
# extended LDIF
#
# LDAPv3
# base <dc=wikimedia,dc=org> (default) with scope subtree
# filter: aRecord=10.68.17.118
# requesting: ALL
#

# ci-jessie-wikimedia-29550.contintcloud.eqiad.wmflabs, hosts, wikimedia.org
dn: dc=ci-jessie-wikimedia-29550.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
 dia,dc=org
aRecord: 10.68.17.118
objectClass: domainRelatedObject
objectClass: dNSDomain
objectClass: puppetClient
objectClass: domain
objectClass: dcObject
objectClass: top
associatedDomain: ci-jessie-wikimedia-29550.contintcloud.eqiad.wmflabs
associatedDomain: ci-jessie-wikimedia-29550.eqiad.wmflabs
l: eqiad
dc: ci-jessie-wikimedia-29550.contintcloud.eqiad.wmflabs
puppetVar: instanceproject=contintcloud
puppetVar: instancename=ci-jessie-wikimedia-29550

# integration-slave-trusty-1021.integration.eqiad.wmflabs, hosts, wikimedia.org
dn: dc=integration-slave-trusty-1021.integration.eqiad.wmflabs,ou=hosts,dc=wik
 imedia,dc=org
aRecord: 10.68.17.118
objectClass: domainRelatedObject
objectClass: dNSDomain
objectClass: puppetClient
objectClass: domain
objectClass: dcObject
objectClass: top
associatedDomain: integration-slave-trusty-1021.integration.eqiad.wmflabs
associatedDomain: integration-slave-trusty-1021.eqiad.wmflabs
l: eqiad
dc: integration-slave-trusty-1021.integration.eqiad.wmflabs
puppetClass: role::ci::slave::labs
puppetVar: instanceproject=integration
puppetVar: instancename=integration-slave-trusty-1021

# integration-slave-trusty-1006.integration.eqiad.wmflabs, hosts, wikimedia.org
dn: dc=integration-slave-trusty-1006.integration.eqiad.wmflabs,ou=hosts,dc=wik
 imedia,dc=org
aRecord: 10.68.17.118
objectClass: domainRelatedObject
objectClass: dNSDomain
objectClass: puppetClient
objectClass: domain
objectClass: dcObject
objectClass: top
associatedDomain: integration-slave-trusty-1006.integration.eqiad.wmflabs
associatedDomain: integration-slave-trusty-1006.eqiad.wmflabs
l: eqiad
dc: integration-slave-trusty-1006.integration.eqiad.wmflabs
puppetClass: role::ci::slave::labs
puppetVar: instanceproject=integration
puppetVar: instancename=integration-slave-trusty-1006

# search result
search: 2
result: 0 Success

# numResponses: 4
# numEntries: 3
krenair@silver:~$ OS_TENANT_NAME=admin nova list --all-tenants | grep "10.68.22.5$"
krenair@silver:~$

(without the $ it returns matches for 10.68.22.56 and 10.68.22.51)

krenair@bastion-01:~$ ldapsearch -x aRecord=10.68.22.5
# extended LDIF
#
# LDAPv3
# base <dc=wikimedia,dc=org> (default) with scope subtree
# filter: aRecord=10.68.22.5
# requesting: ALL
#

# analytics303.analytics.eqiad.wmflabs, hosts, wikimedia.org
dn: dc=analytics303.analytics.eqiad.wmflabs,ou=hosts,dc=wikimedia,dc=org
aRecord: 10.68.22.5
objectClass: domainRelatedObject
objectClass: dNSDomain
objectClass: puppetClient
objectClass: domain
objectClass: dcObject
objectClass: top
associatedDomain: analytics303.analytics.eqiad.wmflabs
associatedDomain: analytics303.eqiad.wmflabs
l: eqiad
dc: analytics303.analytics.eqiad.wmflabs
puppetClass: role::analytics_cluster::client
puppetClass: role::analytics_cluster::hadoop::worker
puppetClass: role::puppet::self
puppetVar: hadoop_cluster_name=analytics-labs-hadoop
puppetVar: hadoop_journalnodes=analytics300.analytics.eqiad.wmflabs,analytics3
 02.analytics.eqiad.wmflabs,analytics303.analytics.eqiad.wmflabs
puppetVar: hadoop_namenodes=analytics300.analytics.eqiad.wmflabs,analytics302.
 analytics.eqiad.wmflabs
puppetVar: puppetmaster=analytics101
puppetVar: instanceproject=analytics
puppetVar: instancename=analytics303

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

Event Timeline

Krenair created this task.Apr 30 2016, 12:46 AM
Krenair renamed this task from LDAP contains two incorrect host entries with aRecord=10.68.17.118 to LDAP contains two extra incorrect host entries with aRecord=10.68.17.118 and one with aRecord=10.68.22.5.Apr 30 2016, 12:49 AM
Krenair updated the task description. (Show Details)

There may be a connection with T115194: Some labs instances IP have multiple PTR entries in DNS here:

krenair@bastion-01:~$ host 10.68.16.97
97.16.68.10.in-addr.arpa domain name pointer ci-jessie-wikimedia-51209.contintcloud.eqiad.wmflabs.
97.16.68.10.in-addr.arpa domain name pointer ci-jessie-wikimedia-52995.contintcloud.eqiad.wmflabs.
97.16.68.10.in-addr.arpa domain name pointer ci-jessie-wikimedia-59269.contintcloud.eqiad.wmflabs.
97.16.68.10.in-addr.arpa domain name pointer ci-jessie-wikimedia-59441.contintcloud.eqiad.wmflabs.
97.16.68.10.in-addr.arpa domain name pointer ci-jessie-wikimedia-49202.contintcloud.eqiad.wmflabs.
97.16.68.10.in-addr.arpa domain name pointer ci-jessie-wikimedia-64225.contintcloud.eqiad.wmflabs.
97.16.68.10.in-addr.arpa domain name pointer tools-worker-1011.tools.eqiad.wmflabs.
97.16.68.10.in-addr.arpa domain name pointer ci-jessie-wikimedia-47624.contintcloud.eqiad.wmflabs.
97.16.68.10.in-addr.arpa domain name pointer ci-jessie-wikimedia-47309.contintcloud.eqiad.wmflabs.
97.16.68.10.in-addr.arpa domain name pointer ci-jessie-wikimedia-60973.contintcloud.eqiad.wmflabs.
97.16.68.10.in-addr.arpa domain name pointer ci-jessie-wikimedia-51020.contintcloud.eqiad.wmflabs.
krenair@bastion-01:~$ ldapsearch -x aRecord=10.68.16.97 | grep dn
dn: dc=ci-jessie-wikimedia-47309.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-47691.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-48076.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-50002.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-51141.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-51523.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-52047.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-52872.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-59701.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-59713.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-60812.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-60973.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-61429.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-62053.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-63183.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-64095.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=ci-jessie-wikimedia-64225.contintcloud.eqiad.wmflabs,ou=hosts,dc=wikime
dn: dc=tools-worker-1011.tools.eqiad.wmflabs,ou=hosts,dc=wikimedia,dc=org

ci-jessie-wikimedia-64225, ci-jessie-wikimedia-47309, and ci-jessie-wikimedia-60973 all appear in both checks

AlexMonk-WMF renamed this task from LDAP contains two extra incorrect host entries with aRecord=10.68.17.118 and one with aRecord=10.68.22.5 to LDAP contains two extra incorrect host entries with aRecord=10.68.17.118, one with aRecord=10.68.22.5, and one with aRecord=10.68.16.120.Aug 9 2016, 1:29 AM
AlexMonk-WMF added a subscriber: AlexMonk-WMF.
root@shinken-01:/etc/shinken# ldapsearch -x aRecord=10.68.16.120
# extended LDIF
#
# LDAPv3
# base <dc=wikimedia,dc=org> (default) with scope subtree
# filter: aRecord=10.68.16.120
# requesting: ALL
#

# deployment-parsoid05.deployment-prep.eqiad.wmflabs, hosts, wikimedia.org
dn: dc=deployment-parsoid05.deployment-prep.eqiad.wmflabs,ou=hosts,dc=wikimedi
 a,dc=org
objectClass: domainrelatedobject
objectClass: dnsdomain
objectClass: domain
objectClass: puppetclient
objectClass: dcobject
objectClass: top
l: eqiad
associatedDomain: i-000006d8.eqiad.wmflabs
associatedDomain: deployment-parsoid05.eqiad.wmflabs
associatedDomain: i-000006d8.deployment-prep.eqiad.wmflabs
associatedDomain: deployment-parsoid05.deployment-prep.eqiad.wmflabs
dc: deployment-parsoid05.deployment-prep.eqiad.wmflabs
aRecord: 10.68.16.120
puppetVar: instancename=deployment-parsoid05
puppetVar: instanceproject=deployment-prep

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

But this host was deleted in T142288#2532425

It's causing deployment-parsoid05 to show up still in http://shinken.wmflabs.org/problems

AlexMonk-WMF added a subscriber: hashar.EditedSep 16 2016, 9:07 PM

deployment-parsoid05 is now gone from LDAP - @hashar created a new instance with that name and deleted it, causing the sink_nova_ldap code to be triggered for the hostname.

Anything from the contintcloud project can be deleted on spot. I am not even sure we need them added in LDAP anyway and maybe we can just skip that provisioning for contintcloud.

Krenair closed this task as Resolved.Nov 2 2018, 3:43 AM
Krenair claimed this task.

These old ou=hosts entries are long gone.

Krenair removed Krenair as the assignee of this task.Nov 2 2018, 3:43 AM