Would it be possible to run navtiming in 'active/active single compute' multi dc setup?

That is:

• navtiming@eqiad consumes from kafka main-eqiad, but only eqiad prefixed topics, and produces to eqiad prometheus.
• navtiming@codfw consumes from kafka main-codfw, but only codfw prefixed topics, and produces to codfw prometheus.

Then there is no need to 'follow' mediawiki. Aggregate metrics can be calculated using thanos.

In T336764#8855608, @Ottomata wrote:

Would it be possible to run navtiming in 'active/active single compute' multi dc setup?

That is:

• navtiming@eqiad consumes from kafka main-eqiad, but only eqiad prefixed topics, and produces to eqiad prometheus.
• navtiming@codfw consumes from kafka main-codfw, but only codfw prefixed topics, and produces to codfw prometheus.

navtiming consumes from kafka-jumbo, I lack the context as to why. But other than that, that's indeed the plan I outlined in the description

navtiming consumes from kafka-jumbo, I lack the context as to why. But other than that, that's indeed the plan I outlined in the description

I think its different? I'm suggesting for navtiming in either DC to consume from kafka main in either DC, their prefixed topics. Because they'd be using different Kafka clusters (main-eqiad or main-codfw), they'd effectively be in distinct 'consumer groups'. If navtiming@eqiad is totally turned off, no eqiad messages would be processed while offline. This could be mitigated by doing a rolling restart of navtiming@eqiad with multiple processes in the same consumer group, as you say.

Basically, make navtiming fully and distinctly multi DC by using Kafka main instead of kafka jumbo-eqiad.

In T336764#8856325, @Ottomata wrote:

navtiming consumes from kafka-jumbo, I lack the context as to why. But other than that, that's indeed the plan I outlined in the description

I think its different? I'm suggesting for navtiming in either DC to consume from kafka main in either DC, their prefixed topics. Because they'd be using different Kafka clusters (main-eqiad or main-codfw), they'd effectively be in distinct 'consumer groups'. If navtiming@eqiad is totally turned off, no eqiad messages would be processed while offline. This could be mitigated by doing a rolling restart of navtiming@eqiad with multiple processes in the same consumer group, as you say.

Note that as of today there's a single navtiming process per site

Basically, make navtiming fully and distinctly multi DC by using Kafka main instead of kafka jumbo-eqiad.

Navtiming consumes from eventlogging_ topics, is that something that is available in kafka-main too? I don't know off hand where/how those topics are generated. At any rate, moving from jumbo to main seems bigger in scope, we can use the single consumer group solution as a stepping stone though!

Navtiming consumes from eventlogging_ topics

OH! right. No that is not available in Kafka main.

Carry on then. :)