Page MenuHomePhabricator

Metrics Platform Integration: PoC for tracking impressions of the Community Updates homepage module
Closed, ResolvedPublic

Description

GrowthExperiments homepage modules already produce analytics events through the Event Platform (HomepageModule schema). This task is to setup instrumentation only for the "Community Updates" module and only for impression events. Any existing property in the existing HomepageModule schema that cannot be sent through Metrics Platform core interaction schemas are not in the scope of the task. However agreeing on schema name and stream name (T370907) would be desired so they don't need to be updated.

Acceptance criteria

  • Create a "new" homepage module schema only using core schemas
  • Setup community updates module to use /analytics/product_metrics/web/base/1.2.0 schema to track impressions
  • Add logging logic to the homepage logging module, ext.growthExperiments.Homepage.Logging, only for the community updates module impressions.
  • Validate events are correctly ingested

Related Objects

Event Timeline

For the purpose of a PoC it is not strictly necessary to create a new schema yet. Using a core interaction schema is sufficient to start ingesting events from one of the homepage modules. Similar as ContentTranslation minT instrumentation (useEventLogging.js#4).

Unless there are objections, we will start using /analytics/product_metrics/web/base/1.2.0.

Change #1059293 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/GrowthExperiments@master] CommunityUpdates: instrument impressions with metrics platform client

https://gerrit.wikimedia.org/r/1059293

This is now ready for review.

A composable-like script useEventLogging.js has been added to wrap the event logging utilizing the new stream and schemas.

The stream is configured (see source) to use a core interaction schema rather than a custom one as discussions about how to capture custom interaction data are still on-going.

Change #1070992 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/EventLogging@master] [DNM] ext.eventLogging: update metrics-platform and expose createMetricsClient

https://gerrit.wikimedia.org/r/1070992

Are we pursuing to use this particular infrastructure? I haven't found any active examples of usage of PageSplitterInstrumentation. Do we know any from the past? @phuedx

Change #1059293 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] CommunityUpdates: instrument impressions with metrics platform client

https://gerrit.wikimedia.org/r/1059293

Etonkovidova subscribed.

Checked in beta - no validation errors were observed; all the needed info seems to be captured (see the example of impression event below; the info in some fields is obscured with dots (...)):

mediawiki.product_metrics.homepage_module_interaction

{"action":"impression","action_source":"community-updates","$schema":"/analytics/product_metrics/web/base/1.2.0","mediawiki":{"database":"cswiki","site_content_language":"cs"},"page":{"content_language":"en"},"agent":{"client_platform":"mediawiki_js","client_platform_family":"desktop_browser"},"performer":{"session_id":"...","active_browsing_session_token":"...","name":"ET34","is_bot":false,"is_logged_in":true,"edit_count_bucket":"5-99 edits","groups":["*","user","autoconfirmed"],"registration_dt":"2020-04-15T18:28:20.000Z","is_temp":false,"language":"en","pageview_id":"a6845a31e3c6773ca5b0"},"sample":{"unit":"pageview","rate":1},"dt":"2024-09-12T23:28:55.599Z","meta":{"stream":"mediawiki.product_metrics.homepage_module_interaction","domain":"cs.wikipedia.beta.wmflabs.org","id":"544111ff-a19c-4ae0-b993-ed839fa79954","dt":"2024-09-12T23:28:58.295Z","request_id":"...","topic":"eqiad.mediawiki.product_metrics.homepage_module_interaction","partition":0,"offset":12},"http":{"request_headers":{"user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36"}}}
nettrom_WMF subscribed.

When verifying these events, I don't see action_context set for these (but it's set for the subsequent click event). Moving it back to "blocked" for now, but that might not be the right column?

When verifying these events, I don't see action_context set for these (but it's set for the subsequent click event). Moving it back to "blocked" for now, but that might not be the right column?

Apologies for the confusion, the scope of this task was only a handshake between metrics platform client and the analytics intake service. The spec is implemented in T374251.

Change #1070992 merged by jenkins-bot:

[mediawiki/extensions/EventLogging@master] Update metrics-platform

https://gerrit.wikimedia.org/r/1070992

Sorry for the noise all. I've made this a subtask of T368326: Update Metrics Platform Client Libraries to accept experiment membership to reflect the fact the blocking relationship – specifically that merging https://gerrit.wikimedia.org/r/1070992 was a blocker.