Page MenuHomePhabricator

Add instrumentation for the Footnote click intent A/B test
Open, Needs TriagePublic

Description

Context

For the experiment we want to track the clicks on the "go to reference" link in the ReferencePreviews. This will be monitored in the A/B test. See Parent ticket

Metadata needed
  • source wiki
  • timestamp
  • logged-in status ( anon / not anon )
  • edit count buckets

(Note:
These are all part of the default web schema. So no need for a custom schema here.

performer_edit_count_bucket must be configured in the test setup UI under Contextual Attributes)

TBD: check what retention metrics are already provided by Test Kitchen. For example, how long the user is on a page before leaving, and whether the same unique user returns to the page.

Events
  • experiment_exposure user visits a page and is either in the control or the treatment group
  • click-footnote-marker user clicks on the footnote marker (to scroll down in the control group, or to pop up reference preview with link in the treatment group)
  • click-footnote-marker-twice user in the treatment group clicks on the same footnote marker for a second time, while the reference preview is open
  • click-toc-link user clicks in the toc to scroll down tot the refrences list
  • click-goto-references user clicks on the Go to references list link in the treatment group
  • initial-pageview-shows-references sent after page render, if the article is short enough that the references list is already visible on page load.
  • scrolled-to-references user has scrolled the page and the references list becomes visible.
  • no-toc-tracking-attached sent when the references TOC tracking could not be attached
Implementation
  • Add tracking logic in ext.cite.track.js
  • Replace current tracking logic from the baseline
  • Create a new unique key for the experiment's name

Event Timeline

Lina_Farid_WMDE added a subscriber: WMDE-Fisch.

@WMDE-Fisch kindly review the "Note" section. Thanks!

WMDE-Fisch renamed this task from Instrumentation for click on "go to reference" link in Reference Previews to Add instrumentation for the Footnote click intend A/B test.Mon, Apr 27, 7:43 AM
WMDE-Fisch updated the task description. (Show Details)
awight renamed this task from Add instrumentation for the Footnote click intend A/B test to Add instrumentation for the Footnote click intent A/B test.Mon, Apr 27, 9:08 AM
awight updated the task description. (Show Details)

Change #1285814 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/extensions/Cite@master] [WIP] Meta-intrument for TOC logic

https://gerrit.wikimedia.org/r/1285814

Change #1285815 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/extensions/Cite@master] [WIP] Switch to a TestKitchen "experiment"

https://gerrit.wikimedia.org/r/1285815

Change #1285814 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] Meta-intrument when TOC logic fails

https://gerrit.wikimedia.org/r/1285814

Change #1285815 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] Switch to a TestKitchen "experiment"

https://gerrit.wikimedia.org/r/1285815

Change #1286380 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/extensions/Cite@master] Remove unnecessary singleton

https://gerrit.wikimedia.org/r/1286380

Change #1286381 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/extensions/Cite@master] Simplify event attachment

https://gerrit.wikimedia.org/r/1286381

Change #1286382 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/extensions/Cite@master] Track vertical scrolling to the references list

https://gerrit.wikimedia.org/r/1286382

Change #1286391 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/extensions/Cite@master] [WIP] Instrument second click on a footnote marker

https://gerrit.wikimedia.org/r/1286391

awight subscribed.

Change #1286380 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] Remove unnecessary singleton

https://gerrit.wikimedia.org/r/1286380

Change #1286381 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] Simplify event attachment

https://gerrit.wikimedia.org/r/1286381

Change #1286382 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] Track vertical scrolling to the references list

https://gerrit.wikimedia.org/r/1286382

Change #1286391 merged by jenkins-bot:

[mediawiki/extensions/Cite@master] Instrument second click on a footnote marker

https://gerrit.wikimedia.org/r/1286391