Page MenuHomePhabricator

Add data-event attributes to table of content links
Closed, ResolvedPublic3 Estimated Story Points

Description

As part of the Table of Contents AB test, we want to measure clicks to the new table of contents vs. the old table of contents.

In order to do this we'll have to modify HTML for both the old & new table of contents and add data-event-{name} attributes to the link elements. The DesktopWebUIActionsTracking schema will then automatically track clicks to these elements.

Tech notes

The new table of contents in Vector is generated via mustache templates, so adding the HTML attribute would be trivial.

The old table of contents however, is generated via the wikitext parser, and adding the HTML attribute in that environment may be risky.

Instead of modifying the server-side rendering of these ToC's, we can instead inject this attribute in the DOM client-side via javascript, which would avoid caching implications as well.

Acceptance criteria

  • Both the old and new table of contents have a data-event attribute on each section link.
  • Clicks on section links are logged via the DesktopWebUIActionsTracking schema with all the data listed here

https://phabricator.wikimedia.org/T302934#7830157
https://phabricator.wikimedia.org/T302934#7830194

QA Results - Beta

ACStatusDetails
1T302934#7830157
2T302934#7830157

QA Results - Prod

ACStatusDetails
1T302934#7830194
2T302934#7830194

Event Timeline

@Jdrewniak - moving to upcoming since analysis was part of the spike, but feel free to move back to needs analysis if necessary

@jwang Do you have a preference for what data-event values I should be adding to the old and new TOC? i.e. data-event-name="ui.toc" and data-event-name="ui.sidebar-toc"? Seems like we have a few naming conventions when it comes to event names so I'm not sure what's best.

Hi @bwang, The keyword naming seems fine to me. If the click events will be captured in DesktopWebUIActionsTracking schema, I want to confirm whether all below event fields will be recorded.

FeatureFields
Old TOCevent.name="ui.toc" & event.action="click" & event.skinversion= xx (1 or 2)
New TOCevent.name="ui.sidebar-toc" & event.action="click" & event.skinversion= xx (1 or 2)

Change 770601 had a related patch set uploaded (by Bernard Wang; author: Bernard Wang):

[mediawiki/skins/Vector@master] Add data-event-name attributes to legacy and sidebar TOC

https://gerrit.wikimedia.org/r/770601

@jwang is the intention that any click to the table of contents will trigger those events e.g. if I click a link inside table of contents or collapse it?

bwang removed bwang as the assignee of this task.Mar 17 2022, 5:06 PM
bwang moved this task from Doing to Code Review on the Web-Team-Backlog (Kanbanana-FY-2021-22) board.
bwang subscribed.

Change 771704 had a related patch set uploaded (by Bernard Wang; author: Bernard Wang):

[mediawiki/extensions/WikimediaEvents@master] Track clicks to spans inside links

https://gerrit.wikimedia.org/r/771704

@jwang is the intention that any click to the table of contents will trigger those events e.g. if I click a link inside table of contents or collapse it?

It's to identify the users who scroll-to-toc for the purpose of clicking the toc. I think we definitely need to record the links inside the toc. But I am not sure how often user will scroll back just to collapse it. What's your hypothesis? Could we record both with different keywords and see what data tells us?

Change 770601 merged by jenkins-bot:

[mediawiki/skins/Vector@master] Add data-event-name attributes to legacy and sidebar TOC

https://gerrit.wikimedia.org/r/770601

@jwang is the intention that any click to the table of contents will trigger those events e.g. if I click a link inside table of contents or collapse it?

It's to identify the users who scroll-to-toc for the purpose of clicking the toc. I think we definitely need to record the links inside the toc. But I am not sure how often user will scroll back just to collapse it. What's your hypothesis? Could we record both with different keywords and see what data tells us?

@jwang I don't think users will be scrolling back just to collapse it. No hypothesis, just trying to make sure whatever engineering solution we land on covers your hypothesis.

Note, the solution we've gone with will log a toc event for clicking the "hide" link as it's also a link in the old table of contents. Not sure if anyone would ever scroll to do that but I think this counts as an interaction so we shouldn't ignore that in our analysis.

image.png (612×304 px, 54 KB)

Test Result - Beta

Status: ✅ PASS
Environment: beta
OS: macOS Monterey
Browser: Chrome
Device: MBP
Emulated Device:NA

Test Artifact(s):

QA Steps

✅ AC1: Both the old and new table of contents have a data-event attribute on each section link.
Per discussion with @Jdrewniak the attribute ended up at the panel level.

Screen Shot 2022-03-31 at 9.32.48 AM.png (396×925 px, 142 KB)

✅ AC2: Clicks on section links are logged via the DesktopWebUIActionsTracking schema with all the data listed here

Screen Shot 2022-03-31 at 9.30.05 AM.png (1×707 px, 283 KB)

Edtadros subscribed.

Test Result - Prod

Status: ✅ PASS
Environment: enwiki
OS: macOS Monterey
Browser: Chrome
Device: MBP
Emulated Device:NA

Test Artifact(s):

QA Steps

✅ AC1: Both the old and new table of contents have a data-event attribute on each section link.
Per discussion with @Jdrewniak the attribute ended up at the panel level.

Screen Shot 2022-04-04 at 3.11.47 PM.png (1×1 px, 241 KB)

Screen Shot 2022-04-04 at 3.14.08 PM.png (1×1 px, 456 KB)

✅ AC2: Clicks on section links are logged via the DesktopWebUIActionsTracking schema with all the data listed here

Screen Shot 2022-04-04 at 3.12.48 PM.png (1×1 px, 409 KB)

Screen Shot 2022-04-04 at 3.31.59 PM.png (1×1 px, 443 KB)

Resolving. Further QA will be done in T305442: QA ToC Instrumentation