Page MenuHomePhabricator

Add experiment enrollment functionality to the Metrics Platform extension
Open, HighPublic5 Estimated Story Points

Description

T370880

Description

Create an api that enrolls users i.e. get enrollment for this user.

  • Query the MPIC experiments api to get buckets (feature variants)
  • Add the bucketing algorithm to assign users by id to a bucket (user will always be in this bucket for the duration of the experiment)
  • There is pre-existing code in WME that provides a similar algorithm based on page id. We can refactor this code to use user id instead.
  • Evenly bucket users based on number of features + control
    • In the Growth team’s use case there is 1 variant so users will be bucketed 50/50 - 0.5 control and 0.5 treatment bucket
    • If there are 2 feature variants, there will be 3 buckets (including control) split into thirds (33% or 0.33)
    • If there are 3 feature variants, there will be 4 buckets (including control) set at 0.25 each

Use the BeforePageDisplay hook to read the api response to fetch the bucket names and ratios, and write to a config variable that will contain the user’s bucket assignments for all experiments the user is enrolled in.

Acceptance Criteria

  • New experiment enrollment API is created
  • Bucketing of users is evenly divided between number of buckets + control
  • Bucketing is based on user id of logged-in users

Required

  • Unit/Integration tests?
  • Documentation?
  • Passed QA?

Event Timeline

Change #1071801 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[mediawiki/extensions/MetricsPlatform@master] Add experiment enrollment functionality

https://gerrit.wikimedia.org/r/1071801

☝️ Being bold. There are a few cosmetic changes that should make the patch easier to test that I can make before merging it…

Is the code from 1071801 directly accessible in beta and production clusters or do we need to enable there as well?

Is the code from 1071801 directly accessible in beta and production clusters or do we need to enable there as well?

I think you've discussed this in Slack with @Sfaci but to answer your question here: Once it's merged, it'll be available on the Beta Cluster and it'll ride the train. All codepaths in the extension default to disabled so we can load it on all wikis so that the code is generally available and only enable it in specific environments.

I rebased https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MetricsPlatform/+/1071801 and responded to a comment from Kosta about excluding temporary accounts. The patch is ready for review again.

I rebased https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MetricsPlatform/+/1071801 and responded to a comment from Kosta about excluding temporary accounts. The patch is ready for review again.

Thanks @phuedx, I will take a look asap. Mind that I'm collecting the Growth feedback for bucketing and will post here before eod but we don't expect that patch to solve it all :)

Change #1071801 merged by jenkins-bot:

[mediawiki/extensions/MetricsPlatform@master] Add experiment enrollment functionality

https://gerrit.wikimedia.org/r/1071801