@KSarabia-WMF can you update as to current status / open questions / next steps?

Open Questions/FYI:

1. Jennifer mentioned she was told that font and dark mode preferences was available on mobile in Minerva skin already. As far as I can see, that's not yet the case unless I'm missing something. @ovasileva can you clarify? Did you mean the OS settings for text size and dark mode preference?
2. In https://phabricator.wikimedia.org/T346979, the baseline is defined as % of pageviews which have set a non-default font size in the Minerva skin (On mobile web). Since users cannot yet set font size preferences, we cannot collect the baseline for this metric yet since it's 100%. On that same note, can we give Jennifer a timeline of when font-size would be deployed?
3. Jennifer can collect the baseline for page preview, limited-width , dark_mode_prepared_by_os preferences, and is_media_viewer_enabled, using the data since Oct 23, when the number of NULL values tapered off.

Summary of data QA for the data collected on Oct 23, 2023

Minerva (Mobile)

Schema: event.MobileWebUIActionsTracking
Instrumentation purpose: collect baseline for % of pageviews which have set a non-default font size in the Minerva skin (On mobile web)

What has been checked	Field name	Status	Note	Snapshot of the result
Skin name	event.skin='minerva'	❓Discussing with engineer	There is no event.skin column in the schema. But see the events are fired in engineer QA (T346106#9244756). Can you confirm whether we plan to capture it in the schema or not? If yes, the field is missing.	.
Font size	event.font	❓Want to confirm the feature availability with PM	Currently the value is ‘0’ or ‘NULL’. ‘0’ value is expected. ‘NULL’ value is not expected. The number of ‘NULL’ is tapering off after the fix on Oct 18, but will take more days. Based on the discussion with the engineer, the feature of font size has not been enabled on mobile. After the deployment the default font size will be 0. The non-default font size will be captured as 2,3.

Vector-2022 (Desktop)

Schema: event.desktopwebuiactionstracking
Instrumentation purpose:

• Collect baselines for
  • % of pageviews which do not use page previews in the Vector 2022 skin (or on desktop overall)
  • % of pageviews which do not use the default width option (and view the site at full width)
  • % of pageviews which a viewing a pinned/unpinned version of the menus on the site (main menu, table of contents, tools menu)
  • % of pageviews which have disabled media viewer

• Capture events for font size feature and dark mode feature.

What has been checked	Field name	Status	Note	snapshot of the result
Font size	event.font	✅ PASS	The number of ‘NULL’ has not completely tapered off after the fix on Oct 18. Based on the discussion with engineer, it’s expect to only have ‘Disable’ value, which means the default font size, as the font feature has not been deployed. After the font feature is enabled on desktop, the non-default font size will be captured as ‘enabled’.
Preview status	event.is_page_preview_on	✅ PASS. Could discard the ‘NULL’ events to calculate the baseline, assuming the ‘NULL’ session will have a similar opt-out rate as the other sessions.	The number of ‘NULL’ has not completely tapered off after the fix on Oct 18. Document the discussion with the engineer here. Default: Is_page_preview_on = true, Non-default: is_page_preview_on=false
Full width status	event.is_full_width	✅ PASS. Could discard the ‘NULL’ events to calculate the baseline, assuming the ‘NULL’ session will have a similar opt-out rate as the other sessions.	The number of ‘NULL’ has not completely tapered off after the fix on Oct 18. Document the discussion with the engineer here. Default: is_full_width =false; Nondefault: is_full_width=true
Pin status of the menus on the site (main menu, table of contents, tools menu)	event.is_pinned	❌	The number of ‘NULL’ has not completely tapered off after the fix on Oct 18. While discussing the default value of event.is_pinned, we discovered that the default value varies between the main menu, table of contents, and tools menu. The default value may change based on the user's screen size. Current pinned rate of logged-in users is 96.88% . The pinned rate of anonymous users in is 99.79%.
Media viewer status	event.is_media_viewer_enabled	✅ PASS. Could discard the ‘NULL’ events to calculate the baseline, assuming the ‘NULL’ session will have a similar opt-out rate as the other sessions.	The number of ‘NULL’ has not completely tapered off after the fix on Oct 18. Default: Is_media_viewer_enabled=true; Non-default: is_media_viewer_enabled=false
OS Dark Mode Setting	event.is_dark_mode_prepared_by_os	✅ PASS. Could discard the ‘NULL’ events to calculate the baseline, assuming the ‘NULL’ session will have a similar opt-out rate as the other sessions.	The number of ‘NULL’ has not completely tapered off after the fix on Oct 18. Default : is_dark_mode_prepared_by_os=false; Non-default: is_dark_mode_prepared_by_os=true
Dark Mode	event.is_dark_mode	✅ PASS	The number of ‘NULL’ has not completely tapered off after the fix on Oct 18. Default: is_dark_mode_on=false; Non-default: is_dark_mode_on=true. It’s expected to only have ‘false’ value as the dark mode feature has not been deployed.	Same as above

@ovasileva , Please, take a look at the summary of the data QA above

@KSarabia-WMF, Could you please confirm two questions from the data QA summary?

Default: Is_media_viewer_enabled=true; Non-default: is_media_viewer_enabled=false

There is no event.skin column in the schema. But see the events are fired in engineer QA (T346106#9244756). Can you confirm whether we plan to capture it in the schema or not? If yes, the field is missing.

There is no event.skin column in the schema.

Ack. I have created a patch for this.

Edit:

event.is_pinned

Need more guidance on how to address this. Let's discuss at upcoming group check-in.

font-size

This is indeed inaccurate for mobile. Created a ticket.

Summary of data QA for the data collected on Nov 8, 2023

Minerva (Mobile)

Schema: event.MobileWebUIActionsTracking
Instrumentation purpose: collect baseline for % of pageviews which have set a non-default font size in the Minerva skin (On mobile web)

What has been checked	Field name	Status	Note	Snapshot of the result
Font size	event.font	✅ PASS	The expected values are: small, regular, large, xlarge. The numbers of '0' and ‘NULL’ are tapering off, but will take more days.