In T391273: navtiming: Loss of Kafka connection fills multiple log files with identical stack traces I tried to use YAML anchors and aliases to configure duplicate settings for the kafka_clusters in deployment-prep. This turned out not to work, but for unclear reasons.
Input:
kafka_clusters: jumbo-deployment-prep: &kafka-jumbo brokers: deployment-kafka-jumbo-5.deployment-prep.eqiad1.wikimedia.cloud: id: 1 rack: A deployment-kafka-jumbo-8.deployment-prep.eqiad1.wikimedia.cloud: id: 2 rack: B deployment-kafka-jumbo-9.deployment-prep.eqiad1.wikimedia.cloud: id: 3 rack: C ipv6: false zookeeper_cluster_name: main-deployment-prep jumbo-eqiad: *kafka-jumbo logging-codfw: brokers: deployment-kafka-logging01.deployment-prep.eqiad1.wikimedia.cloud: id: 1001 rack: B ipv6: false zookeeper_cluster_name: main-deployment-prep logging-eqiad: brokers: deployment-kafka-logging01.deployment-prep.eqiad1.wikimedia.cloud: id: 1001 rack: B ipv6: false zookeeper_cluster_name: main-deployment-prep main-deployment-prep: &kafka-main brokers: deployment-kafka-main-5.deployment-prep.eqiad1.wikimedia.cloud: id: 1 rack: A deployment-kafka-main-6.deployment-prep.eqiad1.wikimedia.cloud: id: 2 rack: B ipv6: false zookeeper_cluster_name: main-deployment-prep main-eqiad: *kafka-main
Stored YAML after Horizon's normalization step:
kafka_clusters: jumbo-deployment-prep: &id001 brokers: deployment-kafka-jumbo-5.deployment-prep.eqiad1.wikimedia.cloud: id: 1 rack: A deployment-kafka-jumbo-8.deployment-prep.eqiad1.wikimedia.cloud: id: 2 rack: B deployment-kafka-jumbo-9.deployment-prep.eqiad1.wikimedia.cloud: id: 3 rack: C ipv6: false zookeeper_cluster_name: main-deployment-prep jumbo-eqiad: *id001 logging-codfw: brokers: deployment-kafka-logging01.deployment-prep.eqiad1.wikimedia.cloud: id: 1001 rack: B ipv6: false zookeeper_cluster_name: main-deployment-prep logging-eqiad: brokers: deployment-kafka-logging01.deployment-prep.eqiad1.wikimedia.cloud: id: 1001 rack: B ipv6: false zookeeper_cluster_name: main-deployment-prep main-deployment-prep: &id002 brokers: deployment-kafka-main-5.deployment-prep.eqiad1.wikimedia.cloud: id: 1 rack: A deployment-kafka-main-6.deployment-prep.eqiad1.wikimedia.cloud: id: 2 rack: B ipv6: false zookeeper_cluster_name: main-deployment-prep main-eqiad: *id002
Failure on use:
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Function Call, Lookup of key 'lookup_options' failed: cloudlib::httpyaml failed Unknown alias: id001 (file: /srv/puppet_code/environments/production/modules/role/manifests/wmcs/instance.pp, line: 3, column: 5) on node deployment-webperf21.deployment-prep.eqiad1.wikimedia.cloud