Patch is available here https://gerrit.wikimedia.org/r/c/operations/software/spicerack/+/723214

Configuration is added in this patch - https://gerrit.wikimedia.org/r/c/operations/puppet/+/721857

@Zbyszko thanks a lot for being the first user of our new process as outlined in https://wikitech.wikimedia.org/wiki/Spicerack#How_to_contribute

New module (Kafka) will introduce the method:

I see no problem for the new module, dependencies are fine.

transfer_kafka_position(self, topics: List[str],from_site: str, from_cluster: str, from_consumer_group: str,to_site: str, to_cluster: str, to_consumer_group: str)

I have some questions and comments for this method:

• Does this method returns anything?
• Would it make sense to rename it to transfer_position given that is part of a Kafka class? The repetition seems unnecessary
• I see that you're passing quite few parameters and they are duplicated between source and destination. I'm wondering if maybe it would make sense to add a small KafkaConsumer (sorry for the name I'm not familiar with Kafka, it probably makes no sense, but it's just to explain my point) @dataclass that has the site, cluster and consumer group. Basically something like:

@dataclass
class KafkaConsumer:
    site: str
    cluster: str
    group: str  # or consumer_group, whatever feels more right in Kafka terms

transfer_position(self, topics: List[str], from: KafkaConsumer, to: KafkaConsumer)

I see also that the proposed patch has now also a set_kafka_position_by_timestamp method, that could benefit from the above dataclass and for which I would too propose to rename it removing the kafka part.

• Does this method returns anything?

No, there's no need - if there are issues, there will be exceptions raised, otherwise it's ok.

• Would it make sense to rename it to transfer_position given that is part of a Kafka class? The repetition seems unnecessary

I like that suggestion, I modified it a little bit so it makes more sense in Kafka context (it was missing from the original as well).

• I see that you're passing quite few parameters and they are duplicated between source and destination.

Makes sense, I modified the code accordingly.