Page MenuHomePhabricator

Puppetize dev cluster change-prop installation
Closed, DeclinedPublic

Description

We maintain a restbase-dev cluster that is used to test new restbase features under conditions similar to production. In order to mimic the environment, there is a dev installation of change-prop, that listens to kafka events from production, samples them according to the size difference between dev and produciton cluster and updates RESTBase.

Right now it's a systemd unit running under RESTBase user and it's not puppetized, making deployment there significantly more complicated then it should be, thus making us neglect updating it.

The installation should be puppetized.

Event Timeline

As a first step, I moved the ChangeProp installs on restbase-dev100[45] to /home/mobrovac/code/change-propagation, removed the SystemD units and re-enabled Puppet there. The next step should be to set up the deployment repo's Scap config, but we have to figure out a way how to do it since each of the two CP instances needs to have a distinctive set of rules and talk to a different Kafka cluster altogether. The latter could be done by exploiting MirrorMaker so as to talk to the same Kafka cluster, but we still have to find a solution to the former problem.

Please let me know if you guys think this is something it can happen this quarter

Please let me know if you guys think this is something it can happen this quarter

I think so, since CP is already puppetised. As I pointed out in my previous comment, we need to figure out how to set up the configs correctly and we should be good to go.

each of the two CP instances needs to have a distinctive set of rules

Why? We can have the same set of rules just like we have in production, some of them will just do nothing, what's the problem?

each of the two CP instances needs to have a distinctive set of rules

Why? We can have the same set of rules just like we have in production, some of them will just do nothing, what's the problem?

Because regardless of MirrorMaker, one config needs to read eqiad.* topics and the other codfw.* ones.