Page MenuHomePhabricator

Log time on page before switching language as part of the UniversalLanguageSelector instrument
Closed, ResolvedPublic3 Estimated Story Points

Description

Readers Web will be A/B testing the existing treatment of the language switcher and a new treatment being worked on as part of Desktop Improvements.

Per T268504, we'd like to understand how long the user was reading the page before changing language via the Universal Language Switcher (the ULS) and how the new treatment impacts that vs the existing treatment.

AC

Developer Notes

WikimediaEvents/modules/ext.wikimediaEvents/universalLanguageSelector.js
// Stephane Bisson's implementation in WikimediaEvents/ext.wikimediaEvents/InukaPageView.js (see
// https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/551259), which I believe
// was based on Readers Web's implementation in the ReadingDepth instrument.

var start = mw.now(),
  hiddenAt = null, timeHidden = 0;

function onHide() {
  if ( !hiddenAt ) {
    hiddenAt = mw.now();
  }
}
function onShow() {
  if ( hiddenAt ) {
    timeHidden += mw.now() - hiddenAt;
    hiddenAt = null;
  }
}

if ( document.hidden ) {
  onHide();
}

document.addEventListener( 'visibilitychange', () => {
  if ( document.hidden ) {
    onHide();
  } else {
    onShow();
  }
} );

function getTimeSinceLoaded() {
  return mw.now() - timePaused;
}

function interfaceLanguageChange( language ) {
  log( {
    /* ... */
    timeOnPage: getTimeOnPage()
  } );
}

QA Results - Beta

ACStatusDetails
1T275794#6994278

QA Results - Prod

ACStatusDetails
1T275794#7001347

Related Objects

StatusSubtypeAssignedTask
OpenMNeisler
Resolvedovasileva
OpenNone
Resolvedalexhollender
Resolvedalexhollender
Resolvedovasileva
Resolvedovasileva
Resolvedovasileva
ResolvedSpikeovasileva
OpenBUG REPORTNone
Resolvedovasileva
Resolvedalexhollender
DeclinedNone
OpenJdlrobson
ResolvedEdtadros
OpenJdlrobson
Declinedovasileva
OpenNone
Opennray
Opennray
Resolvedovasileva
OpenEdtadros
OpenNone
OpenEdtadros
ResolvedEdtadros
ResolvedEdtadros
Openovasileva
OpenNone
DuplicateNone
OpenNone
Resolvedovasileva
OpenNone
OpenMNeisler
Resolvedovasileva

Event Timeline

phuedx added a subscriber: MNeisler.

☝️ per @MNeisler:

If my understanding is correct, using time the page was open might include instances where the page is just open in the background during tabbed browsing. This could skew our results and understanding of how long a user is actually viewing the page before they switch languages.

Jdlrobson added a subscriber: Jdlrobson.

The WikimediaEvents extension now hosts this code.

Jdlrobson set the point value for this task to 3.Mar 10 2021, 5:55 PM

We need to decide where to measure from. The engineers discussed that using performance.timing may be better than mw.now(). we will talk to Megan about how the field is going to be used to determine how to source this value.

If after talking to Megan the engineer feels this needs to be re-estimated they should feel free to move this back to needs analysis

Change 672740 had a related patch set uploaded (by Phuedx; owner: Phuedx):
[schemas/event/secondary@master] universalLanguageSelector: Add timeToChangeLanguage property

https://gerrit.wikimedia.org/r/672740

Change 672742 had a related patch set uploaded (by Phuedx; owner: Phuedx):
[mediawiki/extensions/WikimediaEvents@master] WIP: Add timeToChangeLanguage property to UniversalLanguageSelector events

https://gerrit.wikimedia.org/r/672742

@MNeisler and I discussed this on Thursday, 18th March. We're likely going to be bucketing the timeToChangeLanguage property during analysis and therefore don't need perfect accuracy. The suggested solution is simple and reasonably accurate so I've used it in https://gerrit.wikimedia.org/r/672742.

Change 672740 merged by jenkins-bot:
[schemas/event/secondary@master] universalLanguageSelector: Add timeToChangeLanguage property

https://gerrit.wikimedia.org/r/672740

@phuedx I'm not able to see the event. I'm checking https://en.wikipedia.beta.wmflabs.org/wiki/Barack_Obama. Is there anything I need to do to get it to show up?

Change 672742 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Add timeToChangeLanguage property to UniversalLanguageSelector events

https://gerrit.wikimedia.org/r/672742

@Edtadros: Now that https://gerrit.wikimedia.org/r/672742 has been merged, we should now see the events being logged with the appropriate properties.

Test Result - Beta

Status: ✅ PASS
Environment: beta
OS: macOS Big Sur
Browser: Chrome
Device: MBP
Emulated Device: NA

Test Artifact(s):

QA Steps

When I change language using the ULS, then I see an UniversalLanguageSwitcher event being logged
✅ AC1: The event's timeToChangeLanguage property is set to the number of milliseconds that the page was visible after the instrumentation started to execute{

{
  "version": 2,
  "token": "",
  "contentLanguage": "he",
  "interfaceLanguage": "he",
  "web_session_id": "ede57e482a3a6091ffef",
  "isAnon": false,
  "action": "language-change",
  "context": "interface",
  "selectedInterfaceLanguage": "en",
  "timeToChangeLanguage": 37998.755126953125,
  "userEditBucket": "0 edits"
}
Edtadros added a subscriber: Edtadros.

@ovasileva, This was tested on the current language switcher in production, not the new language button.

Test Result - Prod

Status: ✅ PASS
Environment: enwiki
OS: macOS Big Sur
Browser: Chrome
Device: MBP
Emulated Device: NA

Test Artifact(s):

QA Steps

When I change language using the ULS, then I see an UniversalLanguageSwitcher event being logged
✅ AC1: The event's timeToChangeLanguage property is set to the number of milliseconds that the page was visible after the instrumentation started to execute{

{
  "version": 2,
  "token": "",
  "contentLanguage": "en",
  "interfaceLanguage": "en",
  "web_session_id": "a1f64e09ac54172e877d",
  "isAnon": false,
  "action": "language-change",
  "context": "interface",
  "selectedInterfaceLanguage": "es",
  "timeToChangeLanguage": 19122.804931640625,
  "userEditBucket": "1-4 edits"
}