Project Information

• Name of tool/project: EventStreamConfig
• Project home page: https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/EventStreamConfig/+/master
• Name of team requesting review: Analytics Engineering & Product Infrastructure
• Primary contact: Andrew Otto
• Target date for deployment: January/February 2020
• Link to code repository / patchset: https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/EventStreamConfig/+/master

Description of the tool/project

See README.

Description of how the tool will be used at WMF

• The EventLogging extension will use EventStreamConfig with ResourceLoader to load configs for streams used on certain pages to dynamically configure client stream settings, like sample rate.

• Mobile apps will use the API endpoint to dynamically configure client stream settings like sample rate.

• EventGate event intake service(s) use this to ensure that only events of a specific schema title are allowed into a stream.

• EventBus and other server side event producers uses this to figure out which event intake service a given stream should be produced to.

Dependencies

None.

Has this project been reviewed before?

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/EventStreamConfig/+/545654

Working test environment

In mediawiki-vagrant, clone this extension into mediawiki/extensions. Edit LocalSettings.php and add:

wfLoadExtension('EventStreamConfig');


$wgEventStreams = [
	[
		'stream' => 'test.event',
		'schema_title' => 'test/event',
		'sample_rate' => 0.15,
		'destination' => 'http://10.11.12.13:8192/v1/events',
	],
	[
		'stream' => '/^mediawiki\.job\..+/',
		'schema_title' => 'mediawiki/job',
		'sample_rate' => 0.8,
		'EventServiceName' => 'eventgate-main',
	],
];

You should be able to get stream config from the API endpoint:

curl 'http://dev.wiki.local.wmftest.net:8080/w/api.php?action=streamconfigs&format=json'
curl 'http://dev.wiki.local.wmftest.net:8080/w/api.php?action=streamconfigs&format=json&streams=mediawiki.job.test'

Post-deployment

Analytics Engineering - Andrew Otto