Page MenuHomePhabricator

Update Alt Text ABTestsController logic
Closed, ResolvedPublic

Description

Background

One of our KR's for the Alt Text experiment is to see at least 200 images are improved with Alt text, by at least 50 unique editors, so that we can have the edits evaluated for quality by an accesibility organization. 15-day analysis shows that we are not on track to reach this number in 30 days, so we have extended the length of the experiment to 60 days.

We would like to explore if it's possible to update the logic of the experiment for the second 30 days of the experiment (Oct 5-Nov 5) so that of we can send more users to the experiment (showing the alt text prompt

Task

QA

Can be tested in 7.6.0 (4183)

Event Timeline

Engineering notes:

We have two flags in place that were originally used to prevent users in one experiment from being assigned to another: sawAltTextImageRecommendationsPrompt and sawAltTextArticleEditorPrompt. They continue to act as guards against re-assigning. I tested until I was randomly assigned to the alt text experiment. Then, I updated the percentage to 100%. These guards prevented me from seeing it again, so it would be a one-line change. That would prevent users in the alt-text experiment from being reassigned, but they would also not be assigned to the regular edit experiment.
For users assigned to control, we have logic preventing them from being reassigned to another bucket. The percentage change does not alter that, so these users will not be sorted into an experiment bucket.
It'll work for users who have never entered the experiment logic. For users that already went through it, nothing changes with the percentage update alone, regardless of being in a control group. It would require more logic changes.
Edit: we're ok with users who were assigned in control for one task not being assigned to the test group for the same task.