Page MenuHomePhabricator

deployment-deploy* failing puppet due to kafka not having ipv6
Closed, ResolvedPublic

Description

@Joe added IPv6 support for production Kafka yesterday (https://gerrit.wikimedia.org/r/c/operations/puppet/+/684814), but WMCS does not have IPv6 so Puppet is failing on deployment-deploy*:

taavi@deployment-deploy01:~$ sudo run-puppet-agent
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Function Call, DNS lookup failed for deployment-kafka-jumbo-1.deployment-prep.eqiad1.wikimedia.cloud Resolv::DNS::Resource::IN::AAAA (file: /etc/puppet/modules/profile/manifests/kubernetes/deployment_server/global_config.pp, line: 80, column: 23) on node deployment-deploy01.deployment-prep.eqiad.wmflabs
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

Event Timeline

Change 691494 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] P::kubernetes::deployment_server: Do not use ipv6 on beta

https://gerrit.wikimedia.org/r/691494

Change 737200 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] P::kubernetes: allow disabling kafka ipv6 on hiera

https://gerrit.wikimedia.org/r/737200

Change 691494 abandoned by Majavah:

[operations/puppet@production] P::kubernetes::deployment_server: Do not use ipv6 on beta

Reason:

replaced with less hacky patch https://gerrit.wikimedia.org/r/c/operations/puppet/+/737200/

https://gerrit.wikimedia.org/r/691494

Change 737200 merged by Alexandros Kosiaris:

[operations/puppet@production] P::kubernetes: allow disabling kafka ipv6 on hiera

https://gerrit.wikimedia.org/r/737200

taavi claimed this task.