Page MenuHomePhabricator

EventLogging MEP Upgrade Phase 2 (Sampling)
Closed, ResolvedPublic

Description

Phase 2 (Sampling)
Per-stream uniform sampling rate can be set by a stream configuration field.

As documented in: https://www.mediawiki.org/wiki/Wikimedia_Product/Analytics_Infrastructure/Stream_configuration#Sampling_settings

Event Timeline

Ottomata renamed this task from Eventlogging can use the stream config module to dynamically adjust sampling rates to Eventlogging Client Side can use the stream config module to dynamically adjust sampling rates.Oct 4 2019, 12:31 PM
Milimetric moved this task from Incoming to Event Platform on the Analytics board.

For testing in MediaWiki Vagrant, I had the following to LocalSettings.php:

$wgEventStreams = [
    [
        'stream' => 'analytics.link_hover',
        'schema_title' => 'analytics/link_hover',
    ],
    [
        'stream' => 'pageview',
        'schema_title' => 'analytics/pageview',
        'sampling' => [
        	'rate' => 0.5,
        	'identifier' => 'pageview'
        ],
    ],
    [
        'stream' => 'mobile_apps.session',
        'schema_title' => 'analytics/mobile_apps/session',
        'sampling' => [
        	'identifier' => 'device'
        ],
    ],
];

$wgEventLoggingStreamNames = [
	'analytics.link_hover',
	'pageview',
	'edit',
	'edit.growth',
	'mobile_apps.session',
];

And used the following instrumentation:

( function() {
  $( document ).ready( function() {
    var pageEvent = {
      $schema: '/analytics/pageview/1.0.0',
      page_token: mw.user.getPageviewToken(),
      session_id: mw.user.sessionId()
    };
    mw.eventLog.submit( 'pageview', pageEvent ); // will be sent approx. half the time
    mw.eventLog.submit( 'mobile_apps.session', { $schema: '/analytics/mobile_apps/session/1.0.0' } ); // will not be sent
  });
  // From https://wikitech.wikimedia.org/wiki/Event_Platform/Instrumentation_How_To:
  $( '#content' ).on( 'mouseover', 'a.extiw', function ( jqEvent ) {
    var link = jqEvent.target;
    var linkHoverEvent = {
      // $schema is required and must be set to the exact value of $id that you set in your schema.
      $schema: '/analytics/link_hover/1.0.0',
      link_href: link.href,
      link_title: link.title,
    };
    var streamName = 'analytics.interwiki_link_hover';
    console.log("logging event with data " + JSON.stringify(linkHoverEvent) + " to stream '" + streamName + "'");
    mw.eventLog.submit( streamName, linkHoverEvent );
  } );
}() );

Change 602739 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[mediawiki/extensions/EventLogging@master] Process sampling rules in stream config

https://gerrit.wikimedia.org/r/602739

jlinehan renamed this task from Eventlogging Client Side can use the stream config module to dynamically adjust sampling rates to EventLogging MEP Upgrade Phase 3.Jun 11 2020, 11:30 PM
jlinehan updated the task description. (Show Details)
jlinehan renamed this task from EventLogging MEP Upgrade Phase 3 to EventLogging MEP Upgrade Phase 3 (Sampling).Jun 12 2020, 12:04 AM
mpopov renamed this task from EventLogging MEP Upgrade Phase 3 (Sampling) to EventLogging MEP Upgrade Phase 2 (Sampling).Jun 23 2020, 7:16 PM
mpopov updated the task description. (Show Details)

Change 602739 merged by jenkins-bot:
[mediawiki/extensions/EventLogging@master] Process sampling rules in stream config

https://gerrit.wikimedia.org/r/602739