Page MenuHomePhabricator

[regression-wmf12] new accounts do not get GrowthExperiments features
Open, HighPublicBUG REPORT

Description

  • Create a new account(s) on owikis where GrowthExperiments are enabled.

What happens?:
I tried cswiki, ruwiki and euwiki. @MMiller_WMF created five account on eswiki - none of the new accounts got GrowthExperiments features.

What should have happened instead?: New accounts (at least, some of them) should get the features

Software version (if not a Wikimedia wiki), browser information, screenshots, other information, etc: wmf.12 (eswiki, ruwiki, cswiki`, euwiki).

Event Timeline

Change 702780 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@master] Fix handling of geEnabled flag

https://gerrit.wikimedia.org/r/702780

kostajh triaged this task as High priority.Jul 2 2021, 5:43 AM

This is basically an Unbreak Now! for Growth team, because it means 1) users are not opted into our features, 2) we are unable to collect data for analyzing our features, 3) communities that expect the feature to function a certain way are going to be confused.

I'd love to deploy the one-line fix for this today, since next week folks will be out for the foundation-wide holiday.

Change 702808 had a related patch set uploaded (by Kosta Harlan; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.12] Fix handling of geEnabled flag

https://gerrit.wikimedia.org/r/702808

Change 702780 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Fix handling of geEnabled flag

https://gerrit.wikimedia.org/r/702780

Change 702893 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] selenium: Use Special:CreateAccount for account creation

https://gerrit.wikimedia.org/r/702893

Change 702808 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.37.0-wmf.12] Fix handling of geEnabled flag

https://gerrit.wikimedia.org/r/702808

Mentioned in SAL (#wikimedia-operations) [2021-07-02T09:37:45Z] <tgr@deploy1002> Synchronized php-1.37.0-wmf.12/extensions/GrowthExperiments/includes/HomepageHooks.php: Backport: [[gerrit:702808|Fix handling of geEnabled flag (T285996)]] (duration: 00m 57s)

When checking group assignments for T286002, I noticed that the proportions were off. It took a bit of digging before I remembered this bug. One thing I found when analyzing the proportion of users who were assigned to the Control/Growth features groups from June 28 to 30 is that the probabilities were switched around. 80% of users were assigned to the Control group, 20% of users got the Growth features. The "no users getting the features" is therefore not correct. Leaving this comment here for future reference in case analysis is done based on these groups, so that we know that the proportions are 80/20 Control/Growth from 2021-06-28 to 2021-07-02, and not the expected 20/80.

kostajh moved this task from QA to Code Review on the Growth-Team (Current Sprint) board.

There's still a patch on here that needs code review.