Page MenuHomePhabricator

Disable MirrorMaker for job queue events
Closed, ResolvedPublic

Description

Until we have TLS support for MirrorMaker for cross-dc kafka job replication in order to enable new JobQueue support for private wikis we need to temporarily disable MirrorMaker for jobs.

We don't really use the mirrored jobs right now and the only reason to mirror them is to avoid losing the jobs in case of an actual data center disaster, and as things stand now if an disastrous event occurs losing jobs will not be the biggest of our problems, so right now it's acceptable to just disable mirroring jobs.

Mirroring should be enabled back when we get TLS for kafka and mirror maker (Q1)

Details

Related Gerrit Patches:

Event Timeline

Pchelolo triaged this task as Medium priority.Apr 11 2018, 5:25 PM
Pchelolo created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 11 2018, 5:25 PM

This goes for change-prop too, right? So blacklist .+\.(job|change-prop)\..+?

Pchelolo added a comment.EditedApr 12 2018, 1:19 PM

@Ottomata I think we can disable anything that have /mediawiki\.job/ in there.

The retry topics will contain the original topic names too.

Ok, but we should keep (other?) change-prop topics mirrored?

@Ottomata yes. We're not enabling normal mediawiki events for private wikis because RESTBase or EventStreams are not ready or that.

Change 425824 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Blacklist mediawiki.job topics from cross DC main <-> main Kafka mirroring

https://gerrit.wikimedia.org/r/425824

fdans moved this task from Incoming to Operational Excellence on the Analytics board.
fdans moved this task from Next Up to Ready to Deploy on the Analytics-Kanban board.

Change 425824 merged by Ottomata:
[operations/puppet@production] Blacklist mediawiki.job and change-prop from cross DC main <-> main mirror

https://gerrit.wikimedia.org/r/425824

mobrovac closed this task as Resolved.Apr 16 2018, 7:04 PM

Thank you, @Ottomata !