Add Event Logging to capture users switching from mobile to desktop
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• eigyan
	Jun 17 2022, 4:05 AM

Description

As a system I would like to gather the following when a user clicks a link to toggle from mobile to desktop:

Wikimedia project
Page title
Page namespace
User edit count (can be bucketed; can be local or global)
User groups
User anonymous or logged-in

This data will be gathered using the approach noted in https://phabricator.wikimedia.org/T310045

Developer Local Setup

LocalSettings.php

$wgEventStreams = [
	[
		'stream' => 'desktop_mobile_link_clicks',
		'schema_title' => '/analytics/mediawiki/client/metrics_event',
		'destination_event_service' => 'eventgate-analytics-external',

		// TODO: Determine whether this stream should be sampled
		// 'sample' => [
		//   'unit' => 'pageview',
		//   'rate' => ...,
		// ],

		'producers' => [
			'metrics_platform_client' => [
				'events' => [
					'mediawiki.desktop_link.click',
				],
				'provide_values' => [
					'page_title',
					'page_namespace',
					'performer_edit_count_bucket',
					'performer_groups',
					'performer_is_logged_in',
				],
			],
		],
	],
];

Add the stream to the list of streams that the EventLogging extension knows about:

LocalSettings.php

$wgEventLoggingStreamNames[] = 'desktop_mobile_link_clicks';

Checkout MinervaNueu patch having the new click event

https://gerrit.wikimedia.org/r/c/mediawiki/skins/MinervaNeue/+/812009

Enter mobile browser mode and click on the Desktop footer link

Screen Shot 2022-07-28 at 11.40.32 PM.png (1×780 px, 150 KB)

Observe the event triggered an event log message under event_logging in Docker console:

Screen Shot 2022-07-28 at 11.40.42 PM.png (1×1 px, 329 KB)

The registered event stream is called desktop_mobile_link_clicks

QA Testing Steps

- in one browser tab navigate to https://en.wikipedia.beta.wmflabs.org/.
- Using dev tools switch to a mobile emulator, causing the Mobile web layout to be displayed.
- Click on the Desktop link in the footer. Repeat 4-5 times. Toggling between mobile and vector skins using the "Mobile View" link in the vector skin.
- Navigate to https://stream-beta.wmflabs.org/v2/ui/#/
- Search the event streams for the desktop_mobile_link_clicks stream
Observe the desktop_mobile_link_clicks stream events in the https://stream-beta.wmflabs.org/v2/ui/#/ UI.

Details

Subject	Repo	Branch	Lines +/-
Add metric platfrom event tracking on mobile desktop toggle	mediawiki/skins/MinervaNeue	master	+4 -0
Track switch to desktop clicks	mediawiki/extensions/MobileFrontend	master	+2 -1
[config]: Add click event logging for mobile and desktop	operations/mediawiki-config	master	+20 -0

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
Resolved	Goal	ovasileva	T198313 [GOAL] Advanced mobile contributions
Resolved		Samwalton9-WMF	T208457 Collect data on users who are switching from mobile to desktop
Resolved		• eigyan	T310852 Add Event Logging to capture users switching from mobile to desktop

Event Timeline

• eigyan created this task.Jun 17 2022, 4:05 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 17 2022, 4:05 AM

I removed a note about "and vice versa" because I don't think we're interested in desktop->mobile switches. This can remain scoped to the mobile->desktop toggle.

@Samwalton9 roger that!

• eigyan claimed this task.Jun 21 2022, 1:43 PM

• eigyan moved this task from Ready to In Progress on the Moderator-Tools-Team (Kanban) board.

Samwalton9-WMF added a parent task: T208457: Collect data on users who are switching from mobile to desktop.Jun 29 2022, 3:57 PM

Change 812009 had a related patch set uploaded (by Eigyan; author: Eigyan):

[mediawiki/skins/MinervaNeue@master] Add metric platfrom event tracking on mobile desktop toggle

https://gerrit.wikimedia.org/r/812009

gerritbot added a project: Patch-For-Review.Jul 7 2022, 3:01 PM

Is there any reason we couldn't use the existing desktop click tracking schema be used for this?
https://github.com/wikimedia/mediawiki-extensions-WikimediaEvents/blob/master/modules/ext.wikimediaEvents/desktopWebUIActions.js

I think it covers everything you need (with more) except "user groups", however, I think having user groups inside that schema for other events would be extremely valuable. One thing that is not covered in the description here that may be useful is whether the user is in AMC mode.

If this schema is a good fit, you can register the link for click tracking by marking up the HTML with data-event-name='switch_to_desktop''
https://github.com/wikimedia/mediawiki-extensions-WikimediaEvents/blob/master/modules/ext.wikimediaEvents/mobileWebUIActions.js

@phuedx will user permissions groups be captured by default in the new events schema?

In T310852#8063826, @Jdlrobson wrote:

@phuedx will user permissions groups be captured by default in the new events schema?

Metrics Platform can mix in user groups to the event if you request it. You can request it by adding performer_groups to the producers.metrics_platform_client.provide_values part of the relevant stream configuration (see T310045#8009372, for example).

Change 812391 had a related patch set uploaded (by Eigyan; author: Eigyan):

[operations/mediawiki-config@master] [config]: Add click event logging for mobile and desktop

https://gerrit.wikimedia.org/r/812391

• eigyan moved this task from In Progress to Code review on the Moderator-Tools-Team (Kanban) board.Jul 8 2022, 6:18 PM

@Jdlrobson after chatting with @eigyan, it sounds like we chose this approach in part to give @phuedx some feedback on this metrics approach.

@Samwalton9 some questions that I have after chatting with @Jdlrobson and @eigyan:

Do we have a sampling rate in mind? Are we wanting to track all of these clicks across all projects? Or do we want to use the existing sampling rates established by readers web for their click events? Or something else?
How does the choice of approaches here impact the way things show up for the user looking at the numbers, especially as moderator tools add more data points? I don't have any visibility into what things look like in whatever dashboard they show up on
Do we have any concerns about maintaining our own schema? So far we've only amended schemas that have other maintainers.

Do we have a sampling rate in mind? Are we wanting to track all of these clicks across all projects? Or do we want to use the existing sampling rates established by readers web for their click events? Or something else?

We don't need 100% sampling for this, so I'm happy for this to be sampled if necessary for data volume reasons.

How does the choice of approaches here impact the way things show up for the user looking at the numbers, especially as moderator tools add more data points? I don't have any visibility into what things look like in whatever dashboard they show up on

To the best of my knowledge, which granted is limited, both approaches discussed above will give the same kind of end-user data.

Do we have any concerns about maintaining our own schema? So far we've only amended schemas that have other maintainers.

It certainly sounds like it would be easier to add to an existing schema, but I'm happy to let y'all decide what's best from a technical maintenance perspective.

@eigyan After ruminating on this for a few days, I think we're fine to proceed with the current approach with the provision that we setup the schema with the same sampling rates that are used in the mobile web schema, since that's been working well for us.

In T310852#8066404, @jsn.sherman wrote:

@Samwalton9 some questions that I have after chatting with @Jdlrobson and @eigyan:

Do we have a sampling rate in mind? Are we wanting to track all of these clicks across all projects? Or do we want to use the existing sampling rates established by readers web for their click events? Or something else?

Note well that the following sampling rate that Readers Web have chosen are:

Stream	Logged in?	Unit	Default Rate	Notes
eventlogging_DesktopWebUIActionsTracking	Yes	Pageview	100%	-
eventlogging_DesktopWebUIActionsTracking	No	Pageview	0%	20% on arwiki, bnwiki, frwiki, hewiki, htwiki, jawiki, kowiki, viwiki, zhwiki; 20% on wikis with Desktop Improvements enabled
eventlogging_MobileWebUIActionsTracking	-	Pageview	10%	1% on enwiki; 20% on arwiki, bnwiki, frwiki, hewiki, htwiki, jawiki, kowiki, viwiki, zhwiki

I hope to update the Metrics Platform Clients to include the sampling configuration in the event at the time that the event was added to the submission queue (see T312279: Allow custom sampling rate to be passed at call time)

Do we have any concerns about maintaining our own schema? So far we've only amended schemas that have other maintainers.

If you do end up using the Metrics Platform (MP) to instrument this, then your team doesn't have to maintain a schema. The MP team maintain the monoschema that is used to validate all events dispatched by the MP clients.

@Samwalton9 some questions that I have after chatting with @Jdlrobson and @eigyan:

I'd also be happy to chat about this.

Thanks for the informative response, @phuedx!

eventlogging_MobileWebUIActionsTracking - Pageview 10% 1% on enwiki; 20% on arwiki, bnwiki, frwiki, hewiki, htwiki, jawiki, kowiki, viwiki, zhwiki

was the sampling rate I was imagining starting with

I hope to update the Metrics Platform Clients to include the sampling configuration in the event at the time that the event was added to the submission queue (see T312279: Allow custom sampling rate to be passed at call time)

You'll have to forgive me as I've only dipped my toes into the water of our analytics. Does that mean that after implementation, the sample rate could be set per event in the instrumentation itself for a given stream? In the mean time, can we still configure this per stream per project for MP?

If you do end up using the Metrics Platform (MP) to instrument this, then your team doesn't have to maintain a schema. The MP team maintain the monoschema that is used to validate all events dispatched by the MP clients.

That is excellent news.

I'd also be happy to chat about this.

If it seems like I'm misunderstanding, I will absolutely take you up on that.

In T310852#8070139, @jsn.sherman wrote:

Thanks for the informative response, @phuedx!

eventlogging_MobileWebUIActionsTracking - Pageview 10% 1% on enwiki; 20% on arwiki, bnwiki, frwiki, hewiki, htwiki, jawiki, kowiki, viwiki, zhwiki

was the sampling rate I was imagining starting with

@Jdlrobson please correct me if I'm wrong but I believe this sampling rate is only as low as 1% because the schema tracks all interactions with the top row of buttons in mobile web, which is a pretty large amount of data. I assume that desktop switching is going to have a considerably lower interaction rate? Still, I assume we could start with this and increase if necessary?

Actually the clicks themselves account for a very small amount of the data (most sessions don't click anything). However, we log an event for every bucketed user that says they are eligible for click tracking. This allows us to make statements such as: "1 in 100 sessions a user will click the desktop switch link" (Example, not real data). Does that make sense?

Change 815338 had a related patch set uploaded (by Eigyan; author: Eigyan):

[operations/mediawiki-config@master] [wmf-config]: Undeploy GDI Survey Wave 2

https://gerrit.wikimedia.org/r/815338

• eigyan updated the task description. (Show Details)Jul 29 2022, 3:45 AM

Restricted Application added a subscriber: Masumrezarock100. · View Herald TranscriptJul 29 2022, 3:45 AM

• eigyan updated the task description. (Show Details)Jul 29 2022, 3:52 AM

• eigyan updated the task description. (Show Details)Jul 29 2022, 2:21 PM

• eigyan updated the task description. (Show Details)Jul 29 2022, 3:54 PM

• eigyan updated the task description. (Show Details)Jul 29 2022, 4:15 PM

• eigyan updated the task description. (Show Details)Jul 29 2022, 8:26 PM

• eigyan updated the task description. (Show Details)

• eigyan updated the task description. (Show Details)Aug 1 2022, 4:11 AM

Change 812009 merged by jenkins-bot:

[mediawiki/skins/MinervaNeue@master] Add metric platfrom event tracking on mobile desktop toggle

https://gerrit.wikimedia.org/r/812009

ReleaseTaggerBot added a project: MW-1.39-notes (1.39.0-wmf.25; 2022-08-15).Aug 4 2022, 6:00 PM

Change 812391 merged by jenkins-bot:

[operations/mediawiki-config@master] [config]: Add click event logging for mobile and desktop

https://gerrit.wikimedia.org/r/812391

phuedx mentioned this in T314924: EventLogging queue not being drained on page unload in Chrome.Aug 10 2022, 3:57 PM

• eigyan changed the task status from Open to Stalled.Aug 17 2022, 2:31 PM

Change 826306 had a related patch set uploaded (by Eigyan; author: Eigyan):

[mediawiki/extensions/MobileFrontend@master] WIP: Track switch to desktop clicks

https://gerrit.wikimedia.org/r/826306

We discovered that user groups aren't tracked as part of the mobile tracking at present. We'll follow up with a second ticket for this.

In T310852#8185471, @Samwalton9 wrote:

We discovered that user groups aren't tracked as part of the mobile tracking at present. We'll follow up with a second ticket for this.

Filed at T316230.

Change 826306 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontend@master] Track switch to desktop clicks

https://gerrit.wikimedia.org/r/826306

ReleaseTaggerBot edited projects, added MW-1.39-notes (1.39.0-wmf.27; 2022-08-29); removed MW-1.39-notes (1.39.0-wmf.25; 2022-08-15).Aug 26 2022, 7:00 PM

• eigyan moved this task from Code review to Done on the Moderator-Tools-Team (Kanban) board.Aug 29 2022, 6:06 PM

I'm seeing data coming in as expected :)

Samwalton9-WMF mentioned this in T326212: Improve data logging on Special:Diff and Special:MobileDiff.Jan 5 2023, 12:49 PM

Samwalton9-WMF mentioned this in T353438: What actions do users take on Special:Diff and Special:MobileDiff?.Dec 14 2023, 12:01 PM

Samwalton9-WMF mentioned this in T360923: Remove 'Switch to desktop' event logging.Mar 25 2024, 5:27 PM