Page MenuHomePhabricator

EventLogging MEP Upgrade Phase 2 (Sampling)
Closed, ResolvedPublic

Description

Phase 2 (Sampling)
Per-stream uniform sampling rate can be set by a stream configuration field.

As documented in: https://www.mediawiki.org/wiki/Wikimedia_Product/Analytics_Infrastructure/Stream_configuration#Sampling_settings

Event Timeline

Nuria created this task.Oct 4 2019, 5:24 AM
Ottomata renamed this task from Eventlogging can use the stream config module to dynamically adjust sampling rates to Eventlogging Client Side can use the stream config module to dynamically adjust sampling rates.Oct 4 2019, 12:31 PM
jlinehan moved this task from Triage to MEP on the Better Use Of Data board.
Milimetric triaged this task as High priority.Oct 7 2019, 3:33 PM
Milimetric moved this task from Incoming to Event Platform on the Analytics board.
Ottomata moved this task from Backlog to Next Up on the Event-Platform board.Oct 15 2019, 5:38 PM
jlinehan moved this task from MEP to EPC on the Better Use Of Data board.Nov 19 2019, 5:31 PM
mpopov claimed this task.Jun 5 2020, 4:21 PM
mpopov moved this task from Next Up to In Progress on the Event-Platform board.
mpopov moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
mpopov updated the task description. (Show Details)
mpopov added a comment.Jun 5 2020, 4:53 PM

For testing in MediaWiki Vagrant, I had the following to LocalSettings.php:

$wgEventStreams = [
    [
        'stream' => 'analytics.link_hover',
        'schema_title' => 'analytics/link_hover',
    ],
    [
        'stream' => 'pageview',
        'schema_title' => 'analytics/pageview',
        'sampling' => [
        	'rate' => 0.5,
        	'identifier' => 'pageview'
        ],
    ],
    [
        'stream' => 'mobile_apps.session',
        'schema_title' => 'analytics/mobile_apps/session',
        'sampling' => [
        	'identifier' => 'device'
        ],
    ],
];

$wgEventLoggingStreamNames = [
	'analytics.link_hover',
	'pageview',
	'edit',
	'edit.growth',
	'mobile_apps.session',
];

And used the following instrumentation:

( function() {
  $( document ).ready( function() {
    var pageEvent = {
      $schema: '/analytics/pageview/1.0.0',
      page_token: mw.user.getPageviewToken(),
      session_id: mw.user.sessionId()
    };
    mw.eventLog.submit( 'pageview', pageEvent ); // will be sent approx. half the time
    mw.eventLog.submit( 'mobile_apps.session', { $schema: '/analytics/mobile_apps/session/1.0.0' } ); // will not be sent
  });
  // From https://wikitech.wikimedia.org/wiki/Event_Platform/Instrumentation_How_To:
  $( '#content' ).on( 'mouseover', 'a.extiw', function ( jqEvent ) {
    var link = jqEvent.target;
    var linkHoverEvent = {
      // $schema is required and must be set to the exact value of $id that you set in your schema.
      $schema: '/analytics/link_hover/1.0.0',
      link_href: link.href,
      link_title: link.title,
    };
    var streamName = 'analytics.interwiki_link_hover';
    console.log("logging event with data " + JSON.stringify(linkHoverEvent) + " to stream '" + streamName + "'");
    mw.eventLog.submit( streamName, linkHoverEvent );
  } );
}() );

Change 602739 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[mediawiki/extensions/EventLogging@master] Process sampling rules in stream config

https://gerrit.wikimedia.org/r/602739

jlinehan renamed this task from Eventlogging Client Side can use the stream config module to dynamically adjust sampling rates to EventLogging MEP Upgrade Phase 3.Jun 11 2020, 11:30 PM
jlinehan updated the task description. (Show Details)
jlinehan renamed this task from EventLogging MEP Upgrade Phase 3 to EventLogging MEP Upgrade Phase 3 (Sampling).Jun 12 2020, 12:04 AM
mpopov renamed this task from EventLogging MEP Upgrade Phase 3 (Sampling) to EventLogging MEP Upgrade Phase 2 (Sampling).Jun 23 2020, 7:16 PM
mpopov updated the task description. (Show Details)

Change 602739 merged by jenkins-bot:
[mediawiki/extensions/EventLogging@master] Process sampling rules in stream config

https://gerrit.wikimedia.org/r/602739

mpopov closed this task as Resolved.Aug 4 2020, 12:45 PM