Page MenuHomePhabricator

Create mediawiki_ApiAction Kafka topic
Closed, ResolvedPublic

Description

Create mediawiki_ApiAction Kafka topic with enough partitions to handle processing the full unsampled ApiAction logging feed. This is expected to handle about 5000 log events per second.

Event Timeline

bd808 created this task.Mar 14 2016, 5:18 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 14 2016, 5:18 PM
bd808 assigned this task to Ottomata.Mar 17 2016, 7:38 PM
Ottomata closed this task as Resolved.Mar 17 2016, 7:39 PM
$ kafka topic --create --topic mediawiki_ApiAction --partitions 12 --replication-factor 3
Created topic "mediawiki_ApiAction".


$ kafka topic --describe --topic mediawiki_ApiAction
Topic:mediawiki_ApiAction	PartitionCount:12	ReplicationFactor:3	Configs:
	Topic: mediawiki_ApiAction	Partition: 0	Leader: 22	Replicas: 22,18,20	Isr: 22,18,20
	Topic: mediawiki_ApiAction	Partition: 1	Leader: 12	Replicas: 12,20,22	Isr: 12,20,22
	Topic: mediawiki_ApiAction	Partition: 2	Leader: 13	Replicas: 13,22,12	Isr: 13,22,12
	Topic: mediawiki_ApiAction	Partition: 3	Leader: 14	Replicas: 14,12,13	Isr: 14,12,13
	Topic: mediawiki_ApiAction	Partition: 4	Leader: 18	Replicas: 18,13,14	Isr: 18,13,14
	Topic: mediawiki_ApiAction	Partition: 5	Leader: 20	Replicas: 20,14,18	Isr: 20,14,18
	Topic: mediawiki_ApiAction	Partition: 6	Leader: 22	Replicas: 22,20,12	Isr: 22,20,12
	Topic: mediawiki_ApiAction	Partition: 7	Leader: 12	Replicas: 12,22,13	Isr: 12,22,13
	Topic: mediawiki_ApiAction	Partition: 8	Leader: 13	Replicas: 13,12,14	Isr: 13,12,14
	Topic: mediawiki_ApiAction	Partition: 9	Leader: 14	Replicas: 14,13,18	Isr: 14,13,18
	Topic: mediawiki_ApiAction	Partition: 10	Leader: 18	Replicas: 18,14,20	Isr: 18,14,20
	Topic: mediawiki_ApiAction	Partition: 11	Leader: 20	Replicas: 20,18,22	Isr: 20,18,22