Page MenuHomePhabricator

QA full instrumentation for Reading List experiment
Closed, ResolvedPublic

Description

Requirement

Scope: ReadingLists instrumentation (desktop + mobile).

  • Implement and validate full event instrumentation for the Reading List experiment.
  • All user interactions with the Reading List UI (toolbar, sticky header, popup, and list page) must trigger distinct click or page load events with appropriate attributes.
  • Each event must include consistent contextual data (session, platform, namespace, skin, user state).
  • Instrumentation events must follow the Reading List instrumentation specification as defined in the parent experiment task (T397532).

BDD

Feature: Click tracking for Reading List interactions

  Scenario: Page load event tracking
    Given I am a user in the Reading List experiment
    When I load a page
    Then a `page_load` event is sent with the correct contextual attributes
    And the `action_source` reflects internal or external referer

  Scenario: Save and remove clicks
    Given I am viewing an article in the experiment
    When I click the bookmark icon in the toolbar or sticky header
    Then a `click` event is sent
    And the event includes `action_subtype` and `action_source` matching the UI element clicked
    And the `action_context` contains the current `article_count`

  Scenario: Viewing reading list from top-right icon
    Given I am on an article page
    When I click the top-right reading list icon
    Then a `click` event is sent with `action_subtype: view_reading_list` and `action_source: top_right`

  Scenario: Viewing reading list from article saved popup
    Given I have just saved an article
    When I click “view list” in the saved popup
    Then a `click` event is sent with `action_source: article_saved_popup`

  Scenario: Viewing article from reading list
    Given I am on the Reading List page
    When I click an article link in the list
    Then a `click` event is sent with `action_subtype: view_article`
    And `action_context` includes the total number of saved articles

Test Plan

Test Case 1: Page load event

  1. Log in and enable Reading List experiment.
  2. Load an article page.
  3. Inspect network requests for the instrumentation endpoint.
  4. AC1: action: page_load event is logged.
  5. AC2: action_source is correctly set to internal_referer or external_referer.
  6. AC3: Common contextual attributes are included.

Test Case 2: Save/remove clicks

  1. Click the bookmark icon in the toolbar.
  2. AC4: A click event logs action_subtype: save_to_reading_list and action_source: toolbar.
  3. Click again to remove the article.
  4. AC5: A click event logs action_subtype: remove_from_reading_list and action_source: toolbar.
  5. AC6: action_context includes {article_count: <INT>}.

Test Case 3: View Reading List from top-right

  1. Click the Reading List icon in the top-right corner.
  2. AC7: click event logs action_subtype: view_reading_list and action_source: top_right.

Test Case 4: View Reading List from article saved popup

  1. Save an article, then click “view list” in the popup.
  2. AC8: click event logs action_subtype: view_reading_list and action_source: article_saved_popup.

Test Case 5: View article from Reading List

  1. Navigate to Special:ReadingLists.
  2. Click an article title from the list.
  3. AC9: click event logs action_subtype: view_article and action_source: reading_list.
  4. AC10: action_context contains the total number of saved articles.

QA Results - Prod

Event Timeline

SToyofuku-WMF changed the task status from Open to Stalled.

to be unstalled once all child tasks of the parent (sibling tasks??) are resolved

Jdlrobson-WMF renamed this task from Placeholder for full instrumentation QA to QA full instrumentation for Reading List experiment.Oct 9 2025, 9:48 PM
Jdlrobson-WMF updated the task description. (Show Details)
SToyofuku-WMF changed the task status from Stalled to Open.Nov 6 2025, 8:43 PM

No longer stalled yay

From meeting: you will need to turn off the beta feature and then run mw.xLab.overrideExperimentGroup('we-3-3-4-reading-list-test1', 'treatment'), etc in the console to test this - can be tested on testwiki

(events will appear in the console!)

Test Result - Prod

Status: ✅ PASS
Environment: testwiki
OS: macOS Tahoe 26.x
Browser: Chrome Canary (latest as of test date)
Device: MS
Emulated Device: NA

Test Case 1: Page load event

  1. Log in and enable the Reading List experiment.
  2. Load an article page.
  3. Inspect network requests for the instrumentation endpoint.
  4. AC1: action: page_load event is logged.
  5. AC2: action_source is correctly set to internal_referer or external_referer.
  6. AC3: Common contextual attributes are included.

screenshot 11.png (142×842 px, 25 KB)

screenshot 12.mov.gif (1×1 px, 594 KB)

Test Case 2: Save/remove clicks

  1. Click the bookmark icon in the toolbar.
  2. AC4: A click event logs action_subtype: save_to_reading_list and action_source: toolbar.

The action_subtype is equal to "save_article_to_reading_list". A slight difference that could be attributed to a change in the requirement.

  1. Click again to remove the article.
  2. AC5: A click event logs action_subtype: remove_from_reading_list and action_source: toolbar.
  3. AC6: action_context includes {article_count: <INT>}.

screenshot 12.png (79×1 px, 17 KB)

screenshot 13.mov.gif (1×1 px, 733 KB)

Test Case 3: View Reading List from top-right

  1. Click the Reading List icon in the top-right corner.
  2. AC7: Click event logs action_subtype: view_reading_list and action_source: top_right.

screenshot 13.png (57×1 px, 13 KB)

screenshot 14.mov.gif (1×1 px, 687 KB)

Test Case 4: View Reading List from article saved popup

  1. Save an article, then click “view list” in the popup.
  2. AC8: Click event logs action_subtype: view_reading_list and action_source: article_saved_popup.

screenshot 14.png (61×1 px, 14 KB)

screenshot 15.mov.gif (1×1 px, 959 KB)

Test Case 5: View article from Reading List

  1. Navigate to Special:ReadingLists.
  2. Click an article title from the list.
  3. AC9: Click event logs action_subtype: view_article and action_source: reading_list.
  4. AC10: action_context contains the total number of saved articles.

screenshot 15.png (73×1 px, 16 KB)

screenshot 16.mov.gif (1×1 px, 657 KB)

SToyofuku-WMF claimed this task.

I think the save_to_reading_list vs save_article_to_reading_list discrepancy is fine, since the spec seems to have both - I'll leave a comment in the spec, but we can sign this off 🤞