A couple things are also required for the testing CP to work properly:

1. We need to be able to switch-off event production from change-prop completely by a config stanza in the sys/kafka module
2. We need to be able to configure the prefix for the consumer groups - currently it's static and it's change-prop-${rule.name}. The dev instance of change-prop should have a different prefix to avoid messing with production consumer groups.
3. Currently, when CP starts up, it starts processing where it's left, or if nothing was committed, it fallbacks to the latest offset in each topic via auto.offset.reset=latest property. We've discussed, that going through the backlog of events wouldn't allow us to test properly, because of the do not write if html is the same optimization in RESTBase. This means, to make proper testing we need to follow the life stream of events, but the test-CP wouldn't run continuously, so when we stop and restart it, we don't want it to process backlogs. I propose to switch off commits in the test-CP instance completely via one more config stanza.
4. Switch off log stash logging and metrics.

WIP change-propagation PR.

In T159476#3078304, @Pchelolo wrote:

[ ... ]

4. Switch off log stash logging and metrics.

Do they need to be switched off? Couldn't we just do something to put them under a different namespace?

In T159476#3080233, @Eevans wrote:

In T159476#3078304, @Pchelolo wrote:

[ ... ]

4. Switch off log stash logging and metrics.

Do they need to be switched off? Couldn't we just do something to put them under a different namespace?

Ye, sure, just wanted to highlight that we need not to forget to do something with them to avoid messing with prod/logging

I would be in favour of logging only locally for CP, if even that.

In T159476#3078304, @Pchelolo wrote:

A couple things are also required for the testing CP to work properly:

1. We need to be able to switch-off event production from change-prop completely by a config stanza in the sys/kafka module
2. We need to be able to configure the prefix for the consumer groups - currently it's static and it's change-prop-${rule.name}. The dev instance of change-prop should have a different prefix to avoid messing with production consumer groups.
3. Currently, when CP starts up, it starts processing where it's left, or if nothing was committed, it fallbacks to the latest offset in each topic via auto.offset.reset=latest property. We've discussed, that going through the backlog of events wouldn't allow us to test properly, because of the do not write if html is the same optimization in RESTBase. This means, to make proper testing we need to follow the life stream of events, but the test-CP wouldn't run continuously, so when we stop and restart it, we don't want it to process backlogs. I propose to switch off commits in the test-CP instance completely via one more config stanza.
4. Switch off log stash logging and metrics.

I'm vacillating on the configuration aspect of this a bit, so I'll open it up to some bikeshedding; Do you have any opinions on how all of this should be represented?

I'm not very familiar with change-propagation, but having separate knobs for items 1-3 above seems a little weird. Would it make sense to encapsulate all of this behind a mirror_mode, or dev_mode boolean? Is there any value in having them separately configurable?

I'm not very familiar with change-propagation, but having separate knobs for items 1-3 above seems a little weird. Would it make sense to encapsulate all of this behind a mirror_mode, or dev_mode boolean? Is there any value in having them separately configurable?

Seems like a good idea. No opinion on the switch name, anything will do fine.

@Eevans, is there anything left to do on this task?