Page MenuHomePhabricator

Pass donate experiment group via URL param in Vector 2022
Closed, ResolvedPublic0 Estimated Story Points

Description

Background

The hypothesis that encompasses our donate A/B test is specifically measuring donations (with clickthrough rate as an auxiliary metric), which means we need a way for the fundraising team to distinguish donations that are a part of either treatment or control groups of the experiment. In the case that a user is enrolled in the experiment, we want to modify the wmf_source param (which is currently donate for all donations`) to reflect the experiment group. Note this ticket is for desktop only

Practically speaking, we will be modifying the sitesupport-url message referenced here depending on experiment membership

Refer to the proof of concept in T395944

User story

As a member of the fundraising team, I would like to be able to answer the question of how many donations we received from people who were given the treatment and/or control experiences

Requirements

  • if the user is in the treatment group of the experiment, modify the wmf_source param to be 2025_desktop_test1_treatment
  • if the user is in the control group of the experiment, modify the wmf_source param to be 2025_desktop_test1_control
  • if the user is not in the experiment, the wmf_source param should remain unchanged (donate)

BDD

Feature: Pass donate experiment group via URL param in Vector 2022

  Scenario: Treatment user sees donate link with treatment wmf_source
    Given I am assigned to the treatment variant of the donate experiment
    When I view any Vector 2022 page with the donate link
    Then the donate link URL includes wmf_source=2025_desktop_test1_treatment

  Scenario: Control user sees donate link with control wmf_source
    Given I am assigned to the control variant of the donate experiment
    When I view any Vector 2022 page with the donate link
    Then the donate link URL includes wmf_source=2025_desktop_test1_control

  Scenario: Non-experiment user sees default donate wmf_source
    Given I am not in the donate experiment
    When I view any Vector 2022 page with the donate link
    Then the donate link URL includes wmf_source=donate

Test Steps

Test Case 1: Treatment group modifies wmf_source

  1. Assign treatment variant (e.g., append ?mpo=we-3-2-3-donate-ab-test-1:treatment) and load a Vector 2022 page.
  2. Locate the donate link (from sitesupport-url) in the UI.
  3. Inspect the href.
  4. AC1: Confirm the URL contains wmf_source=2025_desktop_test1_treatment (and retains existing params such as wmf_medium, wmf_campaign, uselang, etc.).

Test Case 2: Control group modifies wmf_source

  1. Assign control variant (e.g., ?mpo=we-3-2-3-donate-ab-test-1:control) and load a Vector 2022 page.
  2. Locate the donate link.
  3. Inspect the href.
  4. AC2: Confirm the URL contains wmf_source=2025_desktop_test1_control (and retains existing params).

Test Case 3: Not in experiment preserves default wmf_source

  1. Load the same page without any experiment assignment parameter (no mpo).
  2. Locate the donate link.
  3. Inspect the href.
  4. AC3: Confirm the URL contains wmf_source=donate (unchanged default), and no experiment-specific value is present.

Test Case 4: Scope limited to desktop Vector 2022

  1. Repeat Test Case 1 or 2 while ensuring Vector 2022 is active (desktop).
  2. Switch to a non-Vector-2022 skin or mobile site, if applicable.
  3. AC4: Confirm the experiment-specific wmf_source change only applies on Vector 2022 (desktop); other contexts are unaffected.

QA Results - Beta

Design

N/A

Acceptance criteria

Either locally or in beta (depending on the status of T395860), we are able to verify that setting experiment membership to either treatment or control has the desired effect. When not in the experiment, the URL remains unchanged

Communication criteria - does this need an announcement or discussion?

N/A

Rollback plan

This should again be fine to revert, with the caveat that we may have to contend with the cache. It is not directly user facing, so worst case we will have to alert the fundraising team and use a different param value if we have to roll out again

Event Timeline

SToyofuku-WMF set the point value for this task to 2.
ovasileva lowered the priority of this task from High to Medium.Jun 23 2025, 5:27 PM

Moving out of sprint to make space for emergent TAHS work

ovasileva raised the priority of this task from Medium to High.Jul 8 2025, 10:41 AM
ovasileva moved this task from Incoming to Backlog on the Reader Experience Team board.
Jdrewniak changed the point value for this task from 2 to 3.Jul 10 2025, 5:09 PM

Change #1155785 had a related patch set uploaded (by LorenMora; author: Stoyofuku-wmf):

[mediawiki/extensions/WikimediaMessages@master] Donate entry point A/B test

https://gerrit.wikimedia.org/r/1155785

Change #1155785 merged by jenkins-bot:

[mediawiki/extensions/WikimediaMessages@master] Add entry point for donate A/B test

https://gerrit.wikimedia.org/r/1155785

Volker_E renamed this task from Pass donate experiment group via URL param (vector) to Pass donate experiment group via URL param in Vector 2022.Aug 4 2025, 6:00 PM
Edtadros subscribed.

Test Result - Beta

Status: ✅ PASS

Test Case 1: Treatment group modifies wmf_source

  1. Assign treatment variant (append ?mpo=we-3-2-3-donate-ab-test-1:treatment) and load a Vector 2022 page.
  2. Locate the donate link (sitesupport-url) in the UI.
  3. Inspect the href.
  4. AC1: Confirm the URL contains wmf_source=2025_desktop_test1_treatment (and retains existing params such as wmf_medium, wmf_campaign, uselang, etc.).

screenshot 18.png (1×1 px, 411 KB)

Test Case 2: Control group modifies wmf_source

  1. Assign control variant (append ?mpo=we-3-2-3-donate-ab-test-1:control) and load a Vector 2022 page.
  2. Locate the donate link.
  3. Inspect the href.
  4. AC2: Confirm the URL contains wmf_source=2025_desktop_test1_control (and retains existing params).

screenshot 19.png (1×1 px, 415 KB)

Test Case 3: Not in experiment preserves default wmf_source

  1. Load the page without any experiment assignment parameter (no mpo).
  2. Locate the donate link.
  3. Inspect the href.
  4. AC3: Confirm the URL contains wmf_source=donate (unchanged default), and no experiment-specific value is present.

screenshot 20.png (1×1 px, 375 KB)

Test Case 4: Scope limited to desktop Vector 2022

  1. Repeat Test Case 1 or 2 ensuring Vector 2022 is active (desktop).
  2. Switch to a non-Vector-2022 skin or mobile site, if applicable.
  3. AC4: Confirm the experiment-specific wmf_source change only applies on Vector 2022 (desktop); other contexts are unaffected.

did a quick spot check. here's vector legacy and minerva.

screenshot 22.png (1×1 px, 382 KB)

screenshot 21.png (1×1 px, 420 KB)

Jdrewniak claimed this task.