Page MenuHomePhabricator

Write donation experiment treatment path for Vector 2022
Closed, ResolvedPublic3 Estimated Story Points

Description

Background

With our new experiment platform, it should be relatively simple for us to gate the new experimental improvements to the donate link behind an experiment check! This ticket encompasses the minimal work for doing so in Vector, calling the same experiment as Minerva, in anticipation of the experiment being turned on gradually

Refer to the proof of concept in T395944 for how to achieve this using user-interface-preferences

Note that to test locally you should use Inssman (or a similar method of modifying headers) to set the X-Experiment-Enrollments header to donate-entry-point-variation=<group>. Ask @SToyofuku-WMF if you have any questions about this!

User story

As a reader of wikipedia, I want to see a heart and button treatment next to the donate link to motivate me to give my $$$

Requirements

  • if a user is in the treatment group of the donate-entry-point-variation experiment, show a heart next to the donate link and render it as a codex button
  • if a user is in the control group or not in the experiment, continue to render the link as is
  • if a user is in either group, a class is added to the button/link: vector-donate-entry-point-variation
  • to begin with nothing should be live anywhere except potentially beta, depending on the status of T395860
Not Required

As part of this ticket, no need to worry about:

  • the color of the button
  • the donate URL
  • instrumentation

Requirement

Gate the Vector 2022 donate entry point behind the donate-entry-point-variation experiment:

  • If the user is in the treatment group, render the donate link as a Codex button and show a heart icon next to the text.
  • If the user is in the control group or not in the experiment, render the donate link as it is today (no Codex button, no heart).
  • If the user is in either group (treatment or control), add the CSS class vector-donate-entry-point-variation to the donate element.
  • Desktop (Vector 2022) only. Nothing should be live outside beta until enabled per T395860.
  • Color of the button, donate URL, and instrumentation are out of scope.

BDD

Feature: Donate entry point treatment path in Vector 2022

  Scenario: Treatment group sees heart and Codex button
    Given I am assigned to donate-entry-point-variation = treatment
    And I am viewing a Vector 2022 page with a donate link
    Then the donate UI renders as a Codex button with a heart icon
    And the donate element has class "vector-donate-entry-point-variation"

  Scenario: Control group sees default donate link
    Given I am assigned to donate-entry-point-variation = control
    And I am viewing a Vector 2022 page with a donate link
    Then the donate UI renders as the default link (no Codex button, no heart)
    And the donate element has class "vector-donate-entry-point-variation"

  Scenario: Not in experiment sees default donate link with no experiment class
    Given I am not enrolled in the donate-entry-point-variation experiment
    And I am viewing a Vector 2022 page with a donate link
    Then the donate UI renders as the default link (no Codex button, no heart)
    And the donate element does not have class "vector-donate-entry-point-variation"

  Scenario: Scope limited to Vector 2022
    Given I view the same page in a non-Vector 2022 skin
    Then the donate UI is unaffected by the experiment treatment

Test Steps

Test Case 1: Treatment group shows heart + Codex button

  1. Assign treatment variant by appending ?mpo=donate-entry-point-variation:treatment to the page URL.
  2. Load any page in Vector 2022 with the donate link.
  3. Inspect the donate UI.
  4. AC1: Donate renders as a Codex button (not a plain link).
  5. AC2: A heart icon appears next to the donate text.
  6. AC3: The donate element includes class vector-donate-entry-point-variation.

Test Case 2: Control group shows default donate link

  1. Assign control variant by appending ?mpo=donate-entry-point-variation:control to the page URL.
  2. Load the same page in Vector 2022.
  3. Inspect the donate UI.
  4. AC4: Donate renders as the current default link (no Codex button).
  5. AC5: No heart icon is present.
  6. AC6: The donate element includes class vector-donate-entry-point-variation.

Test Case 3: Not in experiment shows default donate link with no experiment class

  1. Load the same page without any experiment assignment parameter (no mpo).
  2. Inspect the donate UI.
  3. AC7: Donate renders as the current default link (no Codex button, no heart).
  4. AC8: The donate element does not include class vector-donate-entry-point-variation.

Test Case 4: Scope is limited to Vector 2022

  1. With treatment or control header still set, switch to a non-Vector 2022 skin (e.g., Legacy Vector) and reload.
  2. Inspect the donate UI.
  3. AC9: The experiment treatment does not apply in non-Vector 2022 skins (UI remains unaffected).

QA Results - Beta

Design

image.png (2×2 px, 1 MB)
(from T388149 - color will be handled separately)

Acceptance criteria

The experimental code is added to the WikimediaMessages repo, and it is possible to test the treatment and control experiences, although potentially only locally as this can be signed off before T395860 is done

Nothing is changed in production!!

Communication criteria - does this need an announcement or discussion?

N/A

Rollback plan

Reverting here should be fine. Since there is a caching implication of this, we may have to talk to the Varnish (traffic?) team, but as I mentioned we're explicitly not changing anything in production

Event Timeline

SToyofuku-WMF set the point value for this task to 3.

Presumptuous high priority and democratically elected 3 points

ovasileva lowered the priority of this task from High to Medium.Jun 23 2025, 5:27 PM

Moving out of sprint to make space for emergent TAHS work

ovasileva raised the priority of this task from Medium to High.Jul 8 2025, 10:49 AM
Jdrewniak changed the point value for this task from 3 to 5.Jul 10 2025, 5:00 PM

This work overlaps with T396673 and part of the work has been done in the POC patch for it. So I will pick this up and work on it with T396673.

Change #1155785 had a related patch set uploaded (by LorenMora; author: Stoyofuku-wmf):

[mediawiki/extensions/WikimediaMessages@master] Donate entry point A/B test

https://gerrit.wikimedia.org/r/1155785

Jdrewniak changed the point value for this task from 5 to 3.Jul 29 2025, 6:19 PM

Taking this from you since it's mostly on me to review now

Another experiment patch that shouldn't have any effect in beta/prod yet

Change #1155785 merged by jenkins-bot:

[mediawiki/extensions/WikimediaMessages@master] Add entry point for donate A/B test

https://gerrit.wikimedia.org/r/1155785

Volker_E renamed this task from Write donation experiment treatment path for Vector to Write donation experiment treatment path for Vector 2022.Aug 4 2025, 6:00 PM

Test Result - Beta

Status: ✅ PASS
Environment: beta
OS: macOS Sequoia 15.5
Browser: Chrome Canary (latest as of test date)
Device: MS
Emulated Device: NA

Test Case 1: Treatment group shows heart + Codex button

  1. Assign the treatment variant by appending ?mpo=donate-entry-point-variation:treatment to a Vector 2022 page URL.
  2. Load the page.
  3. AC1: Donate renders as a Codex button (not a plain link).
  4. AC2: A heart icon appears next to the donate text.
  5. AC3: The donate element includes class vector-donate-entry-point-variation.

used mpo=we-3-2-3-donate-ab-test-1:treatment

screenshot 23.png (1×1 px, 398 KB)

Test Case 2: Control group shows default donate link

  1. Assign the control variant by appending ?mpo=donate-entry-point-variation:control to a Vector 2022 page URL.
  2. Load the page.
  3. AC4: Donate renders as the current default link (no Codex button).
  4. AC5: No heart icon is present.
  5. AC6: The donate element includes class vector-donate-entry-point-variation.

used mpo=we-3-2-3-donate-ab-test-1:control

screenshot 24.png (1×1 px, 430 KB)

Test Case 3: Not in experiment shows default donate link with no experiment class

  1. Load the same page without any experiment assignment parameter (no mpo).
  2. Inspect the donate UI.
  3. AC7: Donate renders as the current default link (no Codex button, no heart).
  4. AC8: The donate element does not include class vector-donate-entry-point-variation.

screenshot 25.png (1×1 px, 422 KB)

Test Case 4: Scope limited to Vector 2022

  1. With treatment or control parameter still set, switch to a non-Vector 2022 skin (e.g., Legacy Vector) and reload.
  2. Inspect the donate UI.
  3. AC9: The experiment treatment does not apply in non-Vector 2022 skins (UI remains unaffected).

see T396673#11076357

Edtadros updated the task description. (Show Details)
Edtadros subscribed.
Jdrewniak claimed this task.