We need to create a 'partitioned' topic in both DCs with 8 partitions and other settings similar to all the rest of the topics.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • Pchelolo | T157088 [EPIC] Develop a JobQueue backend based on EventBus | |||
| Resolved | • Pchelolo | T183744 FY17/18 Q3 Program 8 Services Goal: Migrate two high-traffic jobs over to EventBus | |||
| Resolved | • Pchelolo | T185052 Migrate RefreshLinks job to kafka | |||
| Resolved | PRODUCTION ERROR | • Pchelolo | T189204 High (2-3x) write and connection load on enwiki databases | ||
| Resolved | • Pchelolo | T189738 Support per-db-shard concurrency in ChangeProp | |||
| Resolved | Ottomata | T190196 Create <dc>.change-prop.partitioned.mediawiki.job.refreshLinks topic |
Event Timeline
Comment Actions
Done in both eqiad and codfw for both prefixed topics, e.g.:
[@kafka1001:/home/otto] $ kafka topics --describe --topic eqiad.change-prop.partitioned.mediawiki.job.refreshLinks kafka-topics --zookeeper conf1001.eqiad.wmnet,conf1002.eqiad.wmnet,conf1003.eqiad.wmnet/kafka/main-eqiad --describe --topic eqiad.change-prop.partitioned.mediawiki.job.refreshLinks Topic:eqiad.change-prop.partitioned.mediawiki.job.refreshLinks PartitionCount:8 ReplicationFactor:3 Configs: Topic: eqiad.change-prop.partitioned.mediawiki.job.refreshLinks Partition: 0 Leader: 1001 Replicas: 1001,1002,1003 Isr: 1001,1002,1003 Topic: eqiad.change-prop.partitioned.mediawiki.job.refreshLinks Partition: 1 Leader: 1002 Replicas: 1002,1003,1001 Isr: 1002,1003,1001 Topic: eqiad.change-prop.partitioned.mediawiki.job.refreshLinks Partition: 2 Leader: 1003 Replicas: 1003,1001,1002 Isr: 1003,1001,1002 Topic: eqiad.change-prop.partitioned.mediawiki.job.refreshLinks Partition: 3 Leader: 1001 Replicas: 1001,1003,1002 Isr: 1001,1003,1002 Topic: eqiad.change-prop.partitioned.mediawiki.job.refreshLinks Partition: 4 Leader: 1002 Replicas: 1002,1001,1003 Isr: 1002,1001,1003 Topic: eqiad.change-prop.partitioned.mediawiki.job.refreshLinks Partition: 5 Leader: 1003 Replicas: 1003,1002,1001 Isr: 1003,1002,1001 Topic: eqiad.change-prop.partitioned.mediawiki.job.refreshLinks Partition: 6 Leader: 1001 Replicas: 1001,1002,1003 Isr: 1001,1002,1003 Topic: eqiad.change-prop.partitioned.mediawiki.job.refreshLinks Partition: 7 Leader: 1002 Replicas: 1002,1003,1001 Isr: 1002,1003,1