Background

We want to provide the a centralized and consistent way for feature teams to design and run experiments for their features.

See Phase 4.1 of the Data Platform Product Vision and Roadmap

Specific: What do we want to achieve?

1. Build a control plane interface for designing and administering AB and MAB experiments
2. Build client libraries that interact with the control plane on behalf of the Feature Engineer
3. Provide timely feedback about experiment state, including, for example, bucketing rate and bucket sizes

Measurable: How will we know when we've reached our goal?

When a Product Owner can define and deploy an experiment via /wiki/Special:Experiments.

When a Community Member can read the timeline of an experiment on /wiki/Special:Experiments/$experiment.

Achievable: What support will we need to achieve our goal?

• Database for backend
• Review from Product Analytics around the bucketing mechanisms and code

Relevant: Is this goal worthwhile?

Notes

1. I (@phuedx) wrote up some thoughts on the limitations our caching infrastructure places on experimentation on features for logged-out users: https://www.mediawiki.org/wiki/User:Phuedx_(WMF)/Bonus_Exp