Background
Configure and validate event logging in TestKitchen for the Donor badge experiment so we can measure the feature's impact and evaluate success criteria.
This covers registering the experiment, confirming bucket assignment works end-to-end, and verifying that exposure events reach the logging pipeline correctly for all three variants.
Spec events would include (not exhaustive at this time): tapping the donor button, multiple taps per user.
Measurement plan: https://docs.google.com/document/d/177uzZs5WyUNYyQcdD6zGQZ9pWBs0hh5HTXR0B6kbHDY/edit?tab=t.0
Instrumentation spec: https://docs.google.com/spreadsheets/d/1NYyinpkV3dswcamBJjLx0AW8P3KzefQyhD85VITIp7U/edit?gid=1635536298#gid=1635536298
| A: Control | B: Plain message | C: Delightful experience |
Goal
Experiment is registered, bucketing is verified end-to-end, and exposure events are confirmed in the pipeline before any variant-specific UI ships.
Requirements
- Register experiment in TestKitchen with variant identifiers A, B, C (correct weights) and initial traffic split and targeting rules (logged-out, mobile web, post-donation only: donor cookie present).
- Bucket assignment verified as stable and deterministic (persistent at the edge uniques level) across sessions.
- Exposure event schema
- Verify exposure events in the logging pipeline for all three variants using test donor sessions.
QA steps
- Test kitchen will decide if the user is eligible for the experiment and will group the user into control, test A or test B experiment group. On page view we'll fire a page_view event even if they are not a donor
- Donors can only be detected on the client via a call to require( 'ext.wikimediaCustomizations.donor' ).isDonor()
- If the user has donated within the last 250 days then we'll fire an exposure event (This long availability is due to enabled anonymous editing on mobile a while ago)
Depending on the group:
- If the user is a donor in 'control' group A no change
- If the user is a donor and in 'treatment-b-simple' group B they see the donor badge instantly and for subsequent page views and it has no delightful animation
- If the user is a donor in 'treatment-c-delightful' group C they see the donor badge instantly and for subsequent page views and clicking it has delightful animation
Technical information
Experiment Variations (TestKitchen)
- Control (control)
- Treatment B: Simple (treatment-b-simple)
- Treatment C: Delightful (treatment-c-delightful)
Useful JS for testing
$.cookie( 'centralnotice_hide_fundraising', '{"v":1,"created":1779379520,"reason":"donate"}', { path: '/' } ); mw.user.clientPrefs.set( 'minerva-badge', '1' ); mw.testKitchen.overrideExperimentGroup( 'donor-delight-badge', 'control' ); # or mw.testKitchen.overrideExperimentGroup( 'donor-delight-badge', 'treatment-b-simple' ); # or mw.testKitchen.overrideExperimentGroup( 'donor-delight-badge', 'treatment-c-delightful' );
Acceptance Criteria for done
- Experiment registered in TestKitchen with correct targeting and variant weights. https://test-kitchen.wikimedia.org/experiment/donor-delight-badge




