Page MenuHomePhabricator

No event fired for clicking the language button
Closed, ResolvedPublic

Description

We currently have no way to detect when the language button is clicked.

The existing schema has no event for this action.
https://meta.wikimedia.org/wiki/Schema:UniversalLanguageSelector

TODO:

Option 1) A new event is added to the schema (schema change) and we log this
Option 2) We use the DesktopUIClickTracking to capture this event by adding data-event-name="ui.language-btn" See T281928#7063815

QA Results - Beta

ACStatusDetails
1T281928#7115830

QA Results - Prod

ACStatusDetails
1T281928#7115852

Event Timeline

Change 684984 had a related patch set uploaded (by Jdlrobson; author: Phuedx):

[mediawiki/extensions/UniversalLanguageSelector@master] Fire hook when opened via language button

https://gerrit.wikimedia.org/r/684984

ovasileva added a subscriber: ovasileva.
Jdlrobson added a subscriber: MNeisler.

@MNeisler could you advise us here on how this data is going to be used? We assume it will be wanted as part of the UniversalLanguageSelector, but were hoping you could confirm.

@jrobson - Yes, this should be included as part of the UniversalLanguageSelector Schema if possible.

This data will be used in the AB test to determine sessions with clicks to the language button in the header (from users that receive the treatment) and compare to sessions with clicks to the ULS settings (currently logged as event.action = "settings-open" AND event.context = "interlanguage" in ULS) and language links in the sidebar (to be added to ULS as part of T275762 ) from users in the control.

Change 684984 abandoned by Phuedx:

[mediawiki/extensions/UniversalLanguageSelector@master] Fire hook when opened via language button

Reason:

See Nray and Jdlrobson's comments inline.

https://gerrit.wikimedia.org/r/684984

Notes from today's investigation into what is logged and when:

  • When I click the ULS link in the personal tools (either $wgULSPosition = 'personal' in your LocalSettings.php or on commonswiki and wikidatawiki), I see:
    • Almost the same treatment as that triggered by the button in Vector2
    • An event.action = "settings-open" AND event.context = "personal" event logged
  • When I click either the "Display settings" or "Input settings" buttons at the bottom of the ULS, I don't see an event logged

@phuedx the ULS link in personal tools is the UniversalLanguageSelector. The language button we have created in the top right of articles is the "Compact Languages Button". Although they share lots of similiarities they should be considered two different features. In the desktop refresh version of the compact languages button the links to "display settings" and "input settings" at the bottom launch the same think as the button in the personal tools.

Screen Shot 2021-05-07 at 10.08.24 AM.png (736×864 px, 57 KB)

It seems like the solution here is to modify the schema to support this new event to avoid ambiguity betwen the two in analysis. It looks to me like the compact language button feature was never instrumented.

It seems like the solution here is to modify the schema to support this new event to avoid ambiguity betwen the two in analysis. It looks to me like the compact language button feature was never instrumented.

For sure. I just found it noteworthy that the ULS link in the personal tools would trigger an event.action = "settings-open" AND event.context = "personal" event and not, say, event.action = "uls-open" (remembering that clicking the cog in the sidebar also triggers an event.action = "settings-open" AND event.context = "interlanguage" event).

Change 688217 had a related patch set uploaded (by Phuedx; author: Phuedx):

[schemas/event/secondary@master] universalLanguageSelector: Add compact-language-links-open action

https://gerrit.wikimedia.org/r/688217

Change 688218 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/extensions/WikimediaEvents@master] universalLanguageSelector: Log when mw.uls.compact_language_links.open hook fires

https://gerrit.wikimedia.org/r/688218

Change 684984 restored by Phuedx:

[mediawiki/extensions/UniversalLanguageSelector@master] Fire hook when opened via language button

https://gerrit.wikimedia.org/r/684984

ovasileva added a subscriber: phuedx.

Change 688217 merged by jenkins-bot:

[schemas/event/secondary@master] universalLanguageSelector: Add compact-language-links-open action

https://gerrit.wikimedia.org/r/688217

Change 688218 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] universalLanguageSelector: Log when mw.uls.compact_language_links.open hook fires

https://gerrit.wikimedia.org/r/688218

Jdlrobson removed Jdlrobson as the assignee of this task.EditedMay 11 2021, 12:30 AM
Jdlrobson added a subscriber: nray.

The above patches have been merged (thanks @nray for your help here!)

While reviewing we noticed that we are only firing an event for the first click. Is this enough for the purpose of the A/B test?

If we want to capture subsequent clicks, we also have to consider whether the click is to open or to close the language overlay... if the language overlay is open a second click will close it... unless I click outside the overlay and then click the button again ... @MNeisler can you advise?

Moving to "needs more work" but with an inflection on the word "work" to make it a question.

While reviewing we noticed that we are only firing an event for the first click. Is this enough for the purpose of the A/B test?

If we want to capture subsequent clicks, we also have to consider whether the click is to open or to close the language overlay... if the language overlay is open a second click will close it... unless I click outside the overlay and then click the button again ... @MNeisler can you advise?

Good catch. FYI the ULS link in the personal tools fires an event only when the ULS is shown, i.e.

  1. User clicks the ULS link in the personal tools and an event.action = "settings-open" AND event.context = "personal" event is fired
  2. User clicks and the ULS is hidden
  3. User clicks and an event.action = "settings-open" AND event.context = "personal" event is fired
  4. User clicks and the ULS is hidden

The same is true if the user is clicking the cog in the sidebar.

Change 688994 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/extensions/UniversalLanguageSelector@master] Fire hook when compact language links is opened

https://gerrit.wikimedia.org/r/688994

While reviewing we noticed that we are only firing an event for the first click. Is this enough for the purpose of the A/B test?

The first click would be sufficient to determine which ULS treatment the user is receiving for the AB test and compare the number of sessions that include a click to the language button in the header vs sessions that include a click to one of the links/buttons available in the control. With the session id, I should also be able to link users where a click to the language button was recorded with subsequent actions such as switching languages.

However, if feasible, I'd still recommend recording subsequent clicks as well. This would allow us to understand the frequency of usage of the button and be able to directly compare to the usage of ULS setting and language clicks in the control group.

If we want to capture subsequent clicks, we also have to consider whether the click is to open or to close the language overlay... if the language overlay is open a second click will close it... unless I click outside the overlay and then click the button again ... @MNeisler can you advise?

If possible, I think we'd only need to fire an event when the button is opened/shown (not closed) to be consistent with how the other ULS events are instrumented.

Change 688994 merged by jenkins-bot:

[mediawiki/extensions/UniversalLanguageSelector@master] Fire hook when compact language links are opened

https://gerrit.wikimedia.org/r/688994

Thanks @phuedx and @MNeisler .

Note, we made need to backport the above change as part of enabling the A/B test (included in https://phabricator.wikimedia.org/T269093)

Thanks @phuedx and @MNeisler .

Note, we made need to backport the above change as part of enabling the A/B test (included in https://phabricator.wikimedia.org/T269093)

Is this in all cases or only in case we want to begin the A/B test prior to the next train?

Note, we made need to backport the above change as part of enabling the A/B test (included in https://phabricator.wikimedia.org/T269093)

Is this in all cases or only in case we want to begin the A/B test prior to the next train?

Only in the case where we want to begin the A/B test prior to the next train completely rolling out (i.e. EOD next Thursday, 20th May).

Edtadros added a subscriber: Edtadros.

@phuedx The only language selector related event I get is the following one. And it only fires when I select a language. I don't get anything when I open/close the language button. If this is what you are expecting let me know and I'll make a nicely formatted testing comment and table.

{

"event": {
  "version": 2,
  "token": "",
  "contentLanguage": "en",
  "interfaceLanguage": "en",
  "web_session_id": "ceced467437627ce23c4",
  "isAnon": true,
  "action": "compact-language-links-open"
},
"schema": "UniversalLanguageSelector",
"webHost": "en.wikipedia.beta.wmflabs.org",
"wiki": "enwiki",
"$schema": "/analytics/legacy/universallanguageselector/1.3.0",
"client_dt": "2021-05-20T01:49:16.144Z",
"meta": {
  "stream": "eventlogging_UniversalLanguageSelector",
  "domain": "en.wikipedia.beta.wmflabs.org"
}

}

@phuedx The only language selector related event I get is the following one. And it only fires when I select a language. I don't get anything when I open/close the language button. If this is what you are expecting let me know and I'll make a nicely formatted testing comment and table.

...

Could you double check whether that event was fired when you selected a language as I'm seeing a similar event fired when I click the language button (which I what I expect). Here's what I did:

  1. Navigate to https://en.wikipedia.beta.wmflabs.org/wiki/Barack_Obama?useskinversion=2
  2. Click on "1 language" to the right of the title
  3. Observe that a action="compact-language-links-open" event has been logged

@phuedx I see it now. It takes a good number of seconds to appear though.

Screen Shot 2021-05-25 at 6.10.15 AM.png (1×1 px, 563 KB)

Is this good enough to move to Needs QA in Prod?

Is this good enough to move to Needs QA in Prod?

Yes. Being bold and moving it.

Very bold, thanks @phuedx

Test Result - Beta

Status: ✅ PASS
Environment: beta
OS: macOS Big Sur
Browser: Chrome
Device: MBP
Emulated Device: NA

Test Artifact(s):

QA Steps

✅ AC1: An event is fired when the ULS button is clicked
T281928#7111127
T281928#7111487

Screen Shot 2021-05-25 at 6.10.15 AM.png (1×1 px, 563 KB)

Test Result - Prod

Status: ✅ PASS
Environment: enwiki
OS: macOS Big Sur
Browser: Chrome
Device: MBP
Emulated Device: NA

Test Artifact(s):

QA Steps

✅ AC1: An event is fired when the ULS button is clicked

Screen Shot 2021-05-26 at 6.40.24 AM.png (1×1 px, 597 KB)

Edtadros updated the task description. (Show Details)

Change 695351 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/extensions/UniversalLanguageSelector@master] Pass context to compact_language_links.open hook

https://gerrit.wikimedia.org/r/695351

phuedx closed this task as Resolved.EditedMay 27 2021, 4:01 PM
phuedx claimed this task.

Being bold and signing this off and broader instrumentation QA and sign off is happening over at T280770: Instrumentation QA for language switching .