Specific QA Needs:

1. Compare old schema with new schema.
2. Note any discrepancies with volume between old and new.
3. Note any fields that are returning values not consistent with previous values.
4. Note any blockers in data collection and querying.

@KSarabia-WMF, can you also provide the sample rate of the old schema DesktopWebUIActionsTracking? Thanks.

@jwang

'wgWMEDesktopWebUIActionsTracking' => [
	'default' => 0.2, // T258058
	'legacy-vector' => 0,
	'officewiki' => 0,
	'testwiki' => 1, // T256992
	'enwiki' => 0.01,
	// T309260 (some overlap with desktop-improvements group):
	'arwiki' => 0.2,
	'bnwiki' => 0.2,
	'frwiki' => 0.2,
	'hewiki' => 0.2,
	'htwiki' => 0.2,
	'jawiki' => 0.2,
	'kowiki' => 0.2,
	'viwiki' => 0.2,
	'zhwiki' => 0.2,
	// T321734:
	'azwiki' => 0.2,
	'dewiki' => 0.2,
	'eswiki' => 0.2,
	'fawiki' => 0.2,
	'hiwiki' => 0.2,
	'idwiki' => 0.2,
	'itwiki' => 0.2,
	'nlwiki' => 0.2,
	'plwiki' => 0.2,
	'ptwiki' => 0.2,
	'rowiki' => 0.2,
	'ruwiki' => 0.2,
	'thwiki' => 0.2,
	'trwiki' => 0.2,
	'ukwiki' => 0.2,
],

Source

Related conversation happening here regarding discrepancies.

Summary of data QA for the data collected by March 5 2024.

QAed Schemas:
Old: desktopwebuiactionstracking
New: mediawiki_web_ui_actions

What has been checked	Status	Note	Snapshot of the result from the old schema	Snapshot of the result from the new schema
Pick one session_id, compare the result	PASS	Captured same number of events.
Pick one pageview_id, compare the result	PASS	Captured same number of events.
By date	PASS	The new schema captured 0.39% more events than the old schema. The new schema captured 0.34% more sessions than the old schema.
By action	PASS	Between March 1st and Match 5th, the new schema captured 0.18% more click events than old schema. The new schema captured 0.18% more click sessions than old schema. The new schema captured 0.58% more init events than old schema.The new schema captured 0.49% more init sessions than old schema. They are within 2.5% acceptable variance.
By event name	❌	4000+ types of event names in desktopwebuiactionstracking schema schema. Event names contain content info of the pages. . Some event names are in old schema but not in new schema, for example ui.sidebar-toc. Some event names are not in old schema but in new schema, for example, ns=0, most of them are from minerva skin		even_name.diff_comparison
By wiki	PASS	New schema captured 828 wikis, same as the old schema, in the month of Feb 2024.The highest different rate of session count is from small wikis. The events on nowiktionary is 42.3% fewer in new schema. The difference is reduced to 10% since 2024-02-26. The new schema captured 0.85% more events than the old schema in average.The new schema captured 1.44% more sessions than the old schema in average.
By skin name	❓ is it expected that the new schema captured 'minerva' skin with agent.client_platform_family='desktop_browser'.	Based on the data collected from 20240301 to 20240305. The new schema captured 0.52% more vector events than old schema.The new schema captured 0.50% more vector sessions than old schema. The new schema captured 0.3% more vector2022 events than old schema.The new schema captured 0.25% more vector2022 sessions than old schema. minerva skin is not captured in old schema, but captured in new schema with agent.client_platform_family='desktop_browser'. To check with engineer whehter it is expected.
By user type	PASS	Based on the data collected from 20240301 to 20240305. New scheam captured more sessions and events than the old schema, but within 2.5% variance.
agent type	PASS		{F42562439}	{F42562448}
edit count bucket	❓ Is it expected that for logged-out users performer.edit_count_bucket is NULL in new schema, while in old schema, event.editCountBucket is '0 edits'.	For logged-in users, editcountbucket difference is within 2.5% variance. For logged-out users, performer.edit_count_bucket is NULL in new schema, while in old schema, event.editCountBucket is '0 edits'. Need to confirm whether it is expected.
pageNamespace	❌	page.namespace_id is NULL in new schema
viewportSizeBucket	❓	diff is within 2.5% variance. new schema captured 2620 NULL viewportsizebucket with skin minerva . To check with engineer
is_dark_mode_on,	❓ Is null in old schema expected	The diff is within 2.5% variance. old schema captured some NULLs, while new schema did not. To check with engineer
is_dark_mode_prepared_by_os	❓ Is null in old schema expected	The diff is within 2.5% variance. old schema captured some NULLs, while new schema did not. To check with engineer
dark_mode_setting	❓ Is null in old schema expected	The differences in dark_mode_setting being 0, 2, and NULL are within a 2.5% variance. The difference in dark_mode_setting being 1 is larger than 2.5%. Due to the low volume and small absolute difference, we mark it as a pass.
is_full_width	❓	The diff is within a 2.5% variance. The old schema captured some NULLs, while new schema did not.The NULL is from anonymous users. To check with engineer
is_media_viewer_enabled	PASS	The difference is within a 2.5% variance
is_page_preview_on	PASS	The difference is within a 2.5% variance
is_pinned	PASS	The difference is within a 2.5% variance
font	❓	The diff in font=0,1,2 is within a 2.5% variance.. Some values, like large, null, regular and small, are captured in old schema only. To check with engineer.
action_context,	❓ is it expected	value is desktop for minerva skin in new schema
is_bot	❌	performer.is_bot is NULL in new schema	{F42603014}	{F42603033}
sample rate	❌	incorrect in new schema

@KSarabia-WMF , thanks for the info.

The QA summary is posted above (T353029#9622128)
I also documented it in the tab of desktopwebuiactionstracking in the google sheet.

Based on the number of events captured in the old and new schema, we believe the new schema is configured with the same sample rate as the old schema, as mentioned in T353029#9621127. However, it is recorded as 100% for all wikis in the new schema.

SELECT distinct sample
FROM event.mediawiki_web_ui_actions
WHERE year=2024 AND month=3 AND day=8
AND agent.client_platform_family='desktop_browser'

In T353029#9621127, @KSarabia-WMF wrote:

@jwang

'wgWMEDesktopWebUIActionsTracking' => [
	'default' => 0.2, // T258058
	'legacy-vector' => 0,
	'officewiki' => 0,
	'testwiki' => 1, // T256992
	'enwiki' => 0.01,
	// T309260 (some overlap with desktop-improvements group):
	'arwiki' => 0.2,
	'bnwiki' => 0.2,
	'frwiki' => 0.2,
	'hewiki' => 0.2,
	'htwiki' => 0.2,
	'jawiki' => 0.2,
	'kowiki' => 0.2,
	'viwiki' => 0.2,
	'zhwiki' => 0.2,
	// T321734:
	'azwiki' => 0.2,
	'dewiki' => 0.2,
	'eswiki' => 0.2,
	'fawiki' => 0.2,
	'hiwiki' => 0.2,
	'idwiki' => 0.2,
	'itwiki' => 0.2,
	'nlwiki' => 0.2,
	'plwiki' => 0.2,
	'ptwiki' => 0.2,
	'rowiki' => 0.2,
	'ruwiki' => 0.2,
	'thwiki' => 0.2,
	'trwiki' => 0.2,
	'ukwiki' => 0.2,
],

Source

@SToyofuku-WMF and @Mabualruz to review and propose next steps

Change 1013234 had a related patch set uploaded (by Phuedx; author: Phuedx):

[operations/mediawiki-config@master] Update mediawiki.web_ui_actions stream config

https://gerrit.wikimedia.org/r/1013234

In T353029#9622128, @jwang wrote:

❓ is it expected that the new schema captured 'minerva' skin with agent.client_platform_family='desktop_browser'.

Yes. It's possible to use the Minerva skin in a "desktop" environment so I would expect there to be some events like this.

❓ Is it expected that for logged-out users performer.edit_count_bucket is NULL in new schema, while in old schema, event.editCountBucket is '0 edits'.

Yes. This is a discrepancy between the way Web's instrument captures this data and how the Metrics Platform captures the data. See also T329292: Determine what UserBucketService::getUserEditCountBucket should return for anons.

pageNamespace ❌

This is a bug in the JS Metrics Platform client library! See T360609: [JS] Metrics Platform client library doesn't capture page.namespace_id

viewportSizeBucket❓ new schema captured 2620 NULL viewportsizebucket with skin minerva
action_context❓is it expected? value is desktop for minerva skin in new schema

This is a discrepancy between the desktop- and mobile- versions of the instrument. The mobile version of the instrument captures "mobile modes" in the action_context field. If it's not useful any more, then I would recommend against capturing it as it would make merging the two versions of the instrument easier /cc @ovasileva

is_bot ❌ performer.is_bot is NULL in new schema

This is due to a misconfiguration. See https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1013234.

@phuedx Is there anything we can do from the web team side on this?

In T353029#9649357, @Mabualruz wrote:

@phuedx Is there anything we can do from the web team side on this?

At the risk of increasing the scope of the migration, you could consider merging the separate Desktop- and MobileWebUIActionsTracking instruments. The discrepancies between what those two instruments log appear here, i.e.

1. DesktopWebUIActionsTracking ("Desktop") includes the bucketized viewport size but MobileWebUIActionsTracking ("Mobile") doesn't
2. Desktop uses onClickTrack() (source) but Mobile doesn't
3. Mobile tracks "mobile modes" (IIRC whether AMC is enabled) but Desktop doesn't
4. Mobile submits events like action=init,action_source="ns=0" (source) but Desktop submits events like action=init

In T353029#9648729, @phuedx wrote:

In T353029#9622128, @jwang wrote:

is_bot ❌ performer.is_bot is NULL in new schema

This is due to a misconfiguration. See https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1013234.

I've scheduled that patch for deployment on Monday: https://wikitech.wikimedia.org/w/index.php?title=Deployments&diff=2161767&oldid=2161572

We are going to close this because the main goal here has been accomplished.
The ticket to address discrepancies has been written by @SToyofuku-WMF here: T360811
I wrote another Phab ticket that will track the previous comment by @phuedx here: T360955

As a followup, I have documented the sample rate at datahub. @KSarabia-WMF , please review and confirm whether they are reflecting the current configuration.