Page MenuHomePhabricator

Log time on page before switching language as part of the UniversalLanguageSelector instrument
Closed, ResolvedPublic3 Estimated Story Points

Description

Readers Web will be A/B testing the existing treatment of the language switcher and a new treatment being worked on as part of Desktop Improvements (Vector 2022).

Per T268504, we'd like to understand how long the user was reading the page before changing language via the Universal Language Switcher (the ULS) and how the new treatment impacts that vs the existing treatment.

AC

Developer Notes

WikimediaEvents/modules/ext.wikimediaEvents/universalLanguageSelector.js
// Stephane Bisson's implementation in WikimediaEvents/ext.wikimediaEvents/InukaPageView.js (see
// https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/551259), which I believe
// was based on Readers Web's implementation in the ReadingDepth instrument.

var start = mw.now(),
  hiddenAt = null, timeHidden = 0;

function onHide() {
  if ( !hiddenAt ) {
    hiddenAt = mw.now();
  }
}
function onShow() {
  if ( hiddenAt ) {
    timeHidden += mw.now() - hiddenAt;
    hiddenAt = null;
  }
}

if ( document.hidden ) {
  onHide();
}

document.addEventListener( 'visibilitychange', () => {
  if ( document.hidden ) {
    onHide();
  } else {
    onShow();
  }
} );

function getTimeSinceLoaded() {
  return mw.now() - timePaused;
}

function interfaceLanguageChange( language ) {
  log( {
    /* ... */
    timeOnPage: getTimeOnPage()
  } );
}

QA Results - Beta

ACStatusDetails
1T275794#6994278

QA Results - Prod

ACStatusDetails
1T275794#7001347

Related Objects

StatusSubtypeAssignedTask
Resolvedovasileva
Resolvedovasileva
Resolved alexhollender_WMF
Resolved alexhollender_WMF
Resolvedovasileva
Resolvedovasileva
Resolvedovasileva
ResolvedSpikeovasileva
ResolvedBUG REPORTNone
Resolvedovasileva
Resolved alexhollender_WMF
ResolvedPginer-WMF
ResolvedBUG REPORTJdlrobson
ResolvedEdtadros
Resolvedovasileva
Resolvedovasileva
Resolved nray
Resolvedovasileva
ResolvedJdrewniak
Resolvedovasileva
Resolvedphuedx
Resolvedovasileva
Resolvedovasileva
DuplicateNone
Resolvedovasileva
Resolvedovasileva
Declinedovasileva
Resolvedovasileva
Resolvedovasileva
ResolvedEdtadros
ResolvedEdtadros
ResolvedNone
ResolvedBUG REPORTTheklan
Resolvedovasileva
Resolvedovasileva
ResolvedSpikeJdrewniak
OpenSpikeNone
Resolvedjwang
Resolvedjwang
Resolvedovasileva
ResolvedNone
Resolvedjwang
Resolvedovasileva
ResolvedJdlrobson
DuplicateNone
Resolvedovasileva
Resolvedovasileva
Resolvedjwang
Resolvedcjming
Duplicateovasileva
DuplicateNone
DeclinedKrinkle
ResolvedNone
DeclinedJdrewniak
DeclinedNone
Resolvedovasileva
OpenJScherer-WMF
Resolvedabi_
ResolvedPRODUCTION ERRORabi_

Event Timeline

phuedx added a subscriber: MNeisler.

☝️ per @MNeisler:

If my understanding is correct, using time the page was open might include instances where the page is just open in the background during tabbed browsing. This could skew our results and understanding of how long a user is actually viewing the page before they switch languages.

Jdlrobson subscribed.

The WikimediaEvents extension now hosts this code.

Jdlrobson set the point value for this task to 3.Mar 10 2021, 5:55 PM

We need to decide where to measure from. The engineers discussed that using performance.timing may be better than mw.now(). we will talk to Megan about how the field is going to be used to determine how to source this value.

If after talking to Megan the engineer feels this needs to be re-estimated they should feel free to move this back to needs analysis

Change 672740 had a related patch set uploaded (by Phuedx; owner: Phuedx):
[schemas/event/secondary@master] universalLanguageSelector: Add timeToChangeLanguage property

https://gerrit.wikimedia.org/r/672740

Change 672742 had a related patch set uploaded (by Phuedx; owner: Phuedx):
[mediawiki/extensions/WikimediaEvents@master] WIP: Add timeToChangeLanguage property to UniversalLanguageSelector events

https://gerrit.wikimedia.org/r/672742

@MNeisler and I discussed this on Thursday, 18th March. We're likely going to be bucketing the timeToChangeLanguage property during analysis and therefore don't need perfect accuracy. The suggested solution is simple and reasonably accurate so I've used it in https://gerrit.wikimedia.org/r/672742.

Change 672740 merged by jenkins-bot:
[schemas/event/secondary@master] universalLanguageSelector: Add timeToChangeLanguage property

https://gerrit.wikimedia.org/r/672740

@phuedx I'm not able to see the event. I'm checking https://en.wikipedia.beta.wmflabs.org/wiki/Barack_Obama. Is there anything I need to do to get it to show up?

Change 672742 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Add timeToChangeLanguage property to UniversalLanguageSelector events

https://gerrit.wikimedia.org/r/672742

@Edtadros: Now that https://gerrit.wikimedia.org/r/672742 has been merged, we should now see the events being logged with the appropriate properties.

Test Result - Beta

Status: ✅ PASS
Environment: beta
OS: macOS Big Sur
Browser: Chrome
Device: MBP
Emulated Device: NA

Test Artifact(s):

QA Steps

When I change language using the ULS, then I see an UniversalLanguageSwitcher event being logged
✅ AC1: The event's timeToChangeLanguage property is set to the number of milliseconds that the page was visible after the instrumentation started to execute{

{
  "version": 2,
  "token": "",
  "contentLanguage": "he",
  "interfaceLanguage": "he",
  "web_session_id": "ede57e482a3a6091ffef",
  "isAnon": false,
  "action": "language-change",
  "context": "interface",
  "selectedInterfaceLanguage": "en",
  "timeToChangeLanguage": 37998.755126953125,
  "userEditBucket": "0 edits"
}
Edtadros subscribed.

@ovasileva, This was tested on the current language switcher in production, not the new language button.

Test Result - Prod

Status: ✅ PASS
Environment: enwiki
OS: macOS Big Sur
Browser: Chrome
Device: MBP
Emulated Device: NA

Test Artifact(s):

QA Steps

When I change language using the ULS, then I see an UniversalLanguageSwitcher event being logged
✅ AC1: The event's timeToChangeLanguage property is set to the number of milliseconds that the page was visible after the instrumentation started to execute{

{
  "version": 2,
  "token": "",
  "contentLanguage": "en",
  "interfaceLanguage": "en",
  "web_session_id": "a1f64e09ac54172e877d",
  "isAnon": false,
  "action": "language-change",
  "context": "interface",
  "selectedInterfaceLanguage": "es",
  "timeToChangeLanguage": 19122.804931640625,
  "userEditBucket": "1-4 edits"
}