In order to be able to fully get off of the Redis-based JobQueue execution, we need to migrate private wikis to the new EventBus infrastructure.
|operations/mediawiki-config : master||Enable EventBus for job events for all but wikitech wikis|
|mediawiki/extensions/EventBus : wmf/1.31.0-wmf.29||Support per-event-type EventBus enabling configuration.|
|mediawiki/extensions/EventBus : master||Support per-event-type EventBus enabling configuration.|
|Resolved||Pchelolo||T157088 [EPIC] Develop a JobQueue backend based on EventBus|
|Resolved||Pchelolo||T190327 FY17/18 Q4 Program 8 Services Goal: Complete the JobQueue transition to EventBus|
|Resolved||Pchelolo||T191464 Enable CP4JQ support for private wikis|
|Resolved||Ottomata||T192005 Disable MirrorMaker for job queue events|
Technically nothing prevents us from just enabling the new JobQueue for private wikis.
Note that ChangeProp and RESTBase update use-case doesn't support private wikis, but it's a matter of simple config flag to only submit jobs and not normal events.
The only concern I have is that the jobs are replicated to Hadoop and for private wikis they might contain some private information. @Nuria and @Ottomata - is that ok that jobs from private wikis will be replicate to hadoop or should we add some additional logic to filter them out?
Hm, currently the data we import into Hadoop is readable by anyone with a
Hadoop account (not just analytics-privatedata-users) (but overlap of
Hadoop by non privatedata-users is very small). We could change the
permissions on the event database (or even possibly just a few job topic
tables) to be readable only by analytics-privatedata-users.
In general I think this is ok, as we delete this data within the 90 days.
Hm, however, we’re trying to make internal ‘private’ cross DC data all go
over TLS. If we do this, we would want to have TLS enabled for main-eqiad
<-> main-codfw MirrorMaker instance. To do that we need to upgrade main
Hm, however, we’re trying to make internal ‘private’ cross DC data all go over TLS. If we do this, we would want to have TLS enabled for main-eqiad <-> main-codfw MirrorMaker instance. To do that we need to upgrade main Kafkas first.
Hm, that would significantly complicate and delay things. This is a part of our Q4 goal, what's your timeline on upgrading main kafka and enabling TLS for mirror-maker?
Some workarounds are possible, like not mirroring the private wiki events by sending them to special topics with some suffix, but all what comes to mind is pretty ugly.
Timeline for upgrading main is Q4, but MirrorMaker +TLS wasn't in the plan. I don't think we should block your work though (private cross DC data has happened for a long time, we jut have to get incrementally better).
For posterity, in today's JobQueue biweekly meeting we agreed that having unencrypted mirroring of private wikis' data is not acceptable. Given that TLS work for EventBus' MirrorMaker is scheduled for next quarter, we have decided to temporarily switch mirroring off until the TLS work is done. In this way, we can proceed with enabling support for private wikis and start decommissioning the Redis machines tied to the old JobQueue transport mechanism.