[EPIC] Logged-out user A/B test of Donation entry point improvements
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	SToyofuku-WMF
	May 30 2025, 9:56 PM

Description

NOTE: It's possible another epic ticket already exists, in which case we can close this one as a duplicate. I wanted something to thread the tickets I'm about to create under, but wasn't able to find a project/etc to put them under. In general, feel free to edit this as you see fit

Beginning in Q4 and carrying over into Q1, we will be conducting an A/B test using the new experimentation platform, as a means of determining the optimal donation entry points on both mobile and desktop, as well as helping to test the experimentation platform as the first "real" internal users

Requirements

an A/B test is run on a TBD population of anonymous users
on both mobile and desktop
with the primary metric measured being actual donations (quantity) and auxiliary metric being clickthrough rate of the entry point
the data of which is then analyzed to determine the "winning" groups on both platforms
after which the experimental code is removed to make these changes permanent

Design

I'm sure there are more up to date designs somewhere, but the most recent versions I could find are here: T388149

Measurement plan

https://docs.google.com/document/d/1zIVct6RC8hoSCX5yeqsvSI_aFSRARj1JGgwX3pkRC2Q/edit?usp=sharing

Instrumentation spec

https://docs.google.com/spreadsheets/d/1PLbiTF160-jkqqYMN9HBVyXsf5i0FatKyXsFZNO-cSc/edit?usp=sharing

Related Objects
Search...

Status	Assigned	Task
Resolved	SToyofuku-WMF	T395716 [EPIC] Logged-out user A/B test of Donation entry point improvements
Resolved	SToyofuku-WMF	T395718 Set up experiment in xLab
Declined	None	T395860 Configure donate experiment in beta
Resolved	Jdrewniak	T395861 Write donation experiment treatment path for Vector 2022
Resolved	Jdrewniak	T396682 Instrument donate link experiment in Vector 2022
Resolved	jwang	T401501 Update experiments_registry.yaml file to match the experiment name
Declined	None	T395862 Write donation experiment treatment path for Minerva
Resolved	SToyofuku-WMF	T395944 Scope donate A/B test work and onboard onto experiment platform
Resolved	Jdrewniak	T396673 Pass donate experiment group via URL param in Vector 2022
Resolved	Jdrewniak	T396674 Apply progressive styles to the treatment donate button in Vector 2022
Resolved	ovasileva	T397096 Formalize rollout plan
Resolved	mpopov	T397534 Automated analytics for A/B test of Donation entry point improvements
Resolved	Jdrewniak	T399902 Donate A/B: Remove experiment code
Resolved	HFan-WMF	T401714 Roll out Donate (desktop) experiment to English Wikipedia
Resolved	Jdrewniak	T402330 Build proof of concept for mobile donate A/B test

Event Timeline

SToyofuku-WMF created this task.May 30 2025, 9:56 PM

Restricted Application added a project: Epic. · View Herald TranscriptMay 30 2025, 9:56 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

SToyofuku-WMF added a subtask: T395718: Set up experiment in xLab.May 30 2025, 10:11 PM

@SToyofuku-WMF we tend to put epics in Web-Team-Roadmap these days. This also seems to be missing a project tag e.g. FY2024-25 KR 3.1 Content Discovery

KSarabia-WMF moved this task from Incoming to Q4 on the Web-Team board.Jun 2 2025, 5:35 PM

SToyofuku-WMF added a subtask: T395860: Configure donate experiment in beta.Jun 2 2025, 11:52 PM

SToyofuku-WMF added a subtask: T395861: Write donation experiment treatment path for Vector 2022.Jun 3 2025, 12:03 AM

Wasn't able to finish and I'm beginning to identify some pretty critical open questions, but we should at least have some tickets to consider for the sprint Wednesday. Will keep at this and get things estimated/more refined generally tomorrow 🤞

SToyofuku-WMF added a subtask: T395944: Scope donate A/B test work and onboard onto experiment platform.Jun 3 2025, 6:00 PM

SToyofuku-WMF closed subtask T395862: Write donation experiment treatment path for Minerva as Declined.Jun 11 2025, 9:33 PM

SToyofuku-WMF added a subtask: T396673: Pass donate experiment group via URL param in Vector 2022.Jun 11 2025, 10:19 PM

SToyofuku-WMF added a subtask: T396682: Instrument donate link experiment in Vector 2022.Jun 11 2025, 11:22 PM

SToyofuku-WMF removed a subtask: T396682: Instrument donate link experiment in Vector 2022.

SToyofuku-WMF added a subtask: T396674: Apply progressive styles to the treatment donate button in Vector 2022.

Tickets are created and scoped up to the putting it on beta/rollout steps, which will be dependent on our meeting this Monday - we should have some good work to pull into the sprint, though

bwang closed subtask T395944: Scope donate A/B test work and onboard onto experiment platform as Resolved.Jun 17 2025, 9:45 PM

mpopov closed subtask T397534: Automated analytics for A/B test of Donation entry point improvements as Resolved.Jun 20 2025, 9:39 PM

ovasileva triaged this task as High priority.Jun 27 2025, 2:49 PM

ovasileva added a project: Reader Experience Team.

ovasileva removed a project: Web-Team.

ovasileva closed subtask T397096: Formalize rollout plan as Resolved.Jul 7 2025, 11:22 AM

ovasileva moved this task from Incoming to Backlog on the Reader Experience Team board.Jul 8 2025, 9:37 AM

ovasileva moved this task from Backlog to Epics on the Reader Experience Team board.Jul 8 2025, 10:58 AM

SToyofuku-WMF updated the task description. (Show Details)Aug 1 2025, 7:41 PM

SToyofuku-WMF closed subtask T395860: Configure donate experiment in beta as Declined.Aug 5 2025, 3:03 PM

Jdrewniak closed subtask T395861: Write donation experiment treatment path for Vector 2022 as Resolved.Aug 12 2025, 1:46 PM

Jdrewniak closed subtask T396673: Pass donate experiment group via URL param in Vector 2022 as Resolved.