Page MenuHomePhabricator

Disable MirrorMaker for job queue events
Closed, ResolvedPublic


Until we have TLS support for MirrorMaker for cross-dc kafka job replication in order to enable new JobQueue support for private wikis we need to temporarily disable MirrorMaker for jobs.

We don't really use the mirrored jobs right now and the only reason to mirror them is to avoid losing the jobs in case of an actual data center disaster, and as things stand now if an disastrous event occurs losing jobs will not be the biggest of our problems, so right now it's acceptable to just disable mirroring jobs.

Mirroring should be enabled back when we get TLS for kafka and mirror maker (Q1)

Event Timeline

Pchelolo created this task.

This goes for change-prop too, right? So blacklist .+\.(job|change-prop)\..+?

@Ottomata I think we can disable anything that have /mediawiki\.job/ in there.

The retry topics will contain the original topic names too.

Ok, but we should keep (other?) change-prop topics mirrored?

@Ottomata yes. We're not enabling normal mediawiki events for private wikis because RESTBase or EventStreams are not ready or that.

Change 425824 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Blacklist mediawiki.job topics from cross DC main <-> main Kafka mirroring

Change 425824 merged by Ottomata:
[operations/puppet@production] Blacklist mediawiki.job and change-prop from cross DC main <-> main mirror