Page MenuHomePhabricator

Instrument clicks to links in the Languages list in the sidebar
Closed, ResolvedPublic3 Estimated Story Points

Description

Readers Web will be A/B testing the existing treatment of the language switcher and a new treatment being worked on as part of Desktop Improvements. Those users in the control group will be sent the existing treatment.

Per T268504, we'd like to know how many users are clicking on links in the Languages list in the sidebar. In https://phabricator.wikimedia.org/T268504#6796216, we explored measuring this and landed on adding this functionality to the UniversalLanguageSwitcher instrument.

AC

  • When I click on a link in the Languages list in the sidebar, then I see a UniversalLanguageSwitcher event being logged
    1. The event's context property is set to languages-list; and
    2. The event's language property is set to the language code of the wiki being linked to, i.e. the hreflang attribute of the link

QA Steps

Prod

Local

  • Install EventLogging, WikimediaEvents, + the secondary schema (README on @nray's quickstart guide has good instructions)
  • Because of constraints with local dev environments around proper language listing, to fake having the correct markup (i.e. hreflang attribute has value) in the legacy languages list, include the following js snippet on your local MediaWiki:Common.js :
const languages = document.querySelectorAll(".interlanguage-link-target");
var linkTitleArray;

for (var i = 0; i < languages.length; i++) {
  linkTitleArray = languages[i].getAttribute('title').split(':');
  languages[i].setAttribute('hreflang', linkTitleArray[1].toLowerCase());
}

This comment on the patch explains a bit more with screenshots

  • Once you have the eventgate devserver running locally in your terminal (required some finagling with npm errors - basically include:
	"dependencies": {
		"phantomjs-prebuilt": "^2.1.14"
	}

in the package.json file of your local EventLogging repo before running npm -i)

  • Click on one of the legacy languages in the list of a test page (assuming you have content provider enabled and the js injected markup working), and watch the event logging record the event in the terminal (screenshots in associated Gerrit patch).
  • Change preferences to use Legacy Vector, save changes, click on sidebar language links, note that eventgate-devserver does NOT record clicks.

Developer Notes

WikimediaEvents/modules/ext.wikimediaEvents/universalLanguageSelector.js
function interfaceLanguageChange( language, source ) {
  log( {
    /* ... */
    language: language,
    context: source || 'interface'
  } );
}

/* ... */

document.body.addEventListener( 'click', ( event ) => {
  var el = event.target;
  if ( !el.classList.contains( 'interlanguage-link-target' ) ) {
    return;
  }

  mw.hook( 'mw.uls.interface.language.change' ).fire(
    el.attributes.getNamedItem( 'hreflang' ),
    'languages-list'
  );
} );

Related Objects

Event Timeline

Jdlrobson added a subscriber: Jdlrobson.

The WikimediaEvents extension now hosts this code.

While estimating today, we decided we need to clarify with Megan if we are only tracking number of clicks or do we also need to know the number of users clicking the links (and therefore a user identifier becomes important)

While estimating today, we decided we need to clarify with Megan if we are only tracking number of clicks or do we also need to know the number of users clicking the links (and therefore a user identifier becomes important)

hi @MNeisler - per Nick's comment, do number of users clicking or just total clicks need to be tracked for this instrumentation?

@cjming - Thanks for the ping! If possible, it would be useful to have a user identifier to track the number of users. This will help us understand the frequency of usage across users (i.e. Is it just a few users clicking it frequently or a number of users clicking it once?)

Change 681422 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[mediawiki/extensions/WikimediaEvents@master] Add language switching event logging for legacy sidebar.

https://gerrit.wikimedia.org/r/681422

cjming added a subscriber: cjming.

hi @MNeisler - associated patch should be approved/merged shortly. After some discussion with @nray, it appears that the web_session_id for authenticated users persists between opening/quitting tabs within a browser. However when a browser is quit/reopened, a new web_session_id is generated. Let us know if this is problematic for tracking user clicks appropriately - hopefully not.

Change 681422 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] Add language switching event logging for legacy sidebar.

https://gerrit.wikimedia.org/r/681422

hi @MNeisler - associated patch should be approved/merged shortly. After some discussion with @nray, it appears that the web_session_id for authenticated users persists between opening/quitting tabs within a browser. However when a browser is quit/reopened, a new web_session_id is generated. Let us know if this is problematic for tracking user clicks appropriately - hopefully not.

hi @cjming - Thanks for clarifying. I think that's fine for letting us track clicks on a per session basis. The ULS schema used to have an existing token field, which we can use if we want to track clicks per unique user. Does that field still exist? If so, we should have sufficient data for tracking user clicks appropriately.

The ULS schema used to have an existing token field, which we can use if we want to track clicks per unique user. Does that field still exist? If so, we should have sufficient data for tracking user clicks appropriately.

Thanks @MNeisler -- it looks like the token field will be coming through as an empty string per a recent change in the WikimediaEvents extension. So as it is, token will not be a reliable field to track unique user clicks (it was left in for backwards compatibility).

If it's helpful, here's the description (per 1.2.0 ULS schema version) of the web_session_id property that supersedes the token property:

"properties": {
  "web_session_id": {
    "type": "string",
    "minLength": 20,
    "maxLength": 20,
    "pattern": "^[0-9a-z]{20}$",
    "description": "Identifies a web session: a cluster of actions taken by the user on a website within a limited period of time. A session ID is generated the first time it is requested by the instrumentation code, which is usually the first time the user visits the website.\nIn the current implementation, this ID is shared across windows, tabs, and page views in the same browser. The ID is normally regenerated after the browser is shut down; however, if the browser's \"restore previous session\" feature is used when it restarts, the previous ID is retained.\nInteractions across multiple pages in the same web session may be linked together via this identifier.\nInstrumentation code should assign this with mw.user.sessionId()\n"
  },

Hopefully web_session_id suffices for this purpose - do let us know if otherwise.

nray removed nray as the assignee of this task.Apr 22 2021, 7:46 PM

^^ I'm taking my name off this ticket as I've merged the patch related to this ticket yesterday

Since code review is done, should this move to QA or needs more work?

I was waiting to see Megan's response about the web_session_id before moving it over to QA

@MNeisler I'm thinking about the patch associated with this ticket a bit more - can you confirm whether or not we only want to send an event when the user clicks on a sidebar language while on modern vector? I ask because currently the patch we have sends an event when a user clicks on a language in the sidebar for both modern and legacy vector (and possibly other skins) which I assume is undesirable?

Thanks @MNeisler -- it looks like the token field will be coming through as an empty string per a recent change in the WikimediaEvents extension. So as it is, token will not be a reliable field to track unique user clicks (it was left in for backwards compatibility).

@cjming - I'd recommend adding the token field back in as well as keeping the new web_session_id if possible.

I can use the web_session_id to track user activity on a per session basis which will be needed in the analysis but because there is no way to link the session id to a user, we won't be able to determine the number of unique users using the feature over a certain time period. For example, how many distinct users clicked a link in the language list?

Also, per T268504#6796651, we'll need the token field if we want to know the number of new users accessing language switching functionality.

@MNeisler I'm thinking about the patch associated with this ticket a bit more - can you confirm whether or not we only want to send an event when the user clicks on a sidebar language while on modern vector? I ask because currently the patch we have sends an event when a user clicks on a language in the sidebar for both modern and legacy vector (and possibly other skins) which I assume is undesirable?

@nray - Yes, good catch. For the purposes of the AB test, we'll only want to send an event when a user clicks on a sidebar language while in modern. If I'm understanding the instrumentation changes correctly, I don't believe they'll be another way to identify events by users that were in the control group of the AB test vs legacy users.

Another option we might want to consider is including a skin and skinversion field to the schema. This would allow us to still log all of the events but give us the ability to filter out different skin types as needed.

Per discussion with team, token field will remain dropped and rely on web_session_id alone since this instrumentation is more relevant to track anonymous users.

Change 682703 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[mediawiki/extensions/WikimediaEvents@master] Limit instrumentation to legacy sidebar in modern Vector.

https://gerrit.wikimedia.org/r/682703

cjming moved this task from Code Review to QA on the Readers-Web-Backlog (Kanbanana-FY-2020-21) board.
cjming updated the task description. (Show Details)

Change 682703 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] Limit instrumentation to legacy sidebar in modern Vector.

https://gerrit.wikimedia.org/r/682703

@phuedx / @cjming , I moved this to Needs QA In Prod because I thought it had been deployed to Prod. I don't see any events from the language links in the sidebar. Should this go back to the Needs QA column?

Hey, @Edtadros. I've updated the QA Steps - Prod section to reflect changes that were made to the instrument per T275762#7032377 – namely that the instrument should only log when the user is using the Vector V2 skin. My apologies for the confusion.

Moving to needs more work based on issues found in T280770#7110028

Please could you clarify which issue (or issues) in that comment are relevant to this task?

@phuedx - apologies, this is the wrong task T281928: No event fired for clicking the language button that could potentially need more work, but added a clarifying question to T280770: Instrumentation QA for language switching to confirm. This one I think is ready for signoff per T280770#7110028

Edtadros added a subscriber: Edtadros.

dont mind me...just removing my face....

phuedx closed this task as Resolved.EditedMay 27 2021, 4:01 PM
phuedx claimed this task.

Being bold and signing this off and broader instrumentation QA and sign off is happening over at T280770: Instrumentation QA for language switching .