Page MenuHomePhabricator

Create analytics-test-eqiad zookeeper cluster
Closed, DeclinedPublic

Description

Currently we do not have a zookeeper cluster that is readily available for test purposes within the analytics network.

Currently this means that our analytics-test-hadoop cluster is using our production zookeeper cluster: link

hadoop_clusters:
  analytics-test-hadoop:
    zookeeper_cluster_name: analytics-eqiad

I discovered this problem whilst setting up alluxio for HA.
There is a test-eqiad zookeeper cluster: link
...but this host is outside the analytics networks. It seems to be in use only for the test-eqiad kafka cluster.

There is also a druid-analytics-test-eqiad: link
..but this cluster is dedicated to druid, which is how we have it set up in production.

In order to make the test environment more like the production environment I think it would therefore be useful to set up another zookeeper cluster for the analytics servers' use, named analytics-test-eqiad and ensure that this is what is used for hadoop etc.

Event Timeline

Ottomata subscribed.

Could do this, or allow Analytics VLAN to access test-eqiad ZK cluster. Priority low, unless we need to test a zookeeper specific upgrade and don't want to impact the prod clusters.

This is no longer necessary, now that the analytics vlan firewall has gone. We can now use the test-eqiad zookeeper cluster from the analytics vlan if we wish.