Change #1084935 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[mediawiki/extensions/MetricsPlatform@master] Allow users to override experiment enrollment

https://gerrit.wikimedia.org/r/1084935

hi @phuedx I think I was thrown off by the GrowthExperiments' implementation of overriding bucketing assignments. They use UserOptionsManager to set variants for their users. And actually just very recently Growth team introduced a new class ExperimentUserDefaultsManager which leverages $wgConditionalUserOptions in order to apply variants to existing users without having to store that info in the database. I had started down a path that saves variants to user properties but it got really hairy and after miles of spaghetti code, I scrapped it in favor of a simpler approach.

I may be misunderstanding the assignment here but if the requirement is to provide the ability to force a bucket assignment for a given user, I err'd on the side of checking for a query parameter within the onBeforePageDisplay hook (wrapped in a conditional that checks for a config var that enables this force variant overriding feature) to force a bucketing assignment for a user within a given experiment rather than emulate Growth Team's implementations which include writing/reading to/from the user properties table or using conditional user options. Please advise if I got this totally wrong and we should in fact be saving variants to user properties (my intuition is that this is a bad idea per user_properties table bloat).

Based on my reading of https://phabricator.wikimedia.org/T374471#10221407, I have some questions based on some assumptions:

• I opted to read a query parameter over a JS utility -- do we need to provide both?
• Do we need to support logged-out users? The current implementation in MP only applies to logged-in users -- and since the Growth Team's user cohort seems to be users with accounts, I'm guessing we could use temp accounts and try to save jsconfigvars for those users. Do you think this should be included in the scope of MPIC alpha?
• Do we need to provide custom segmentation for accounts by date or assign sampling rates by platform? This will increase complexity significantly for both MPIC and the MP extension - I'm wondering if this is in scope for this task.

cc @Sfaci

Just shadowing here, @cjming your intuition on the user_properties table bloat concern is correct. That's why in GE we opted to use conditional defaults in ExperimentUserDefaultsManager, because that allows us to target all accounts in a wiki and does not introduce new rows in the properties table. This has also the advantage of being able to use the user options API as the "getter" of a variant. As things are now, our implementation of the force variant assignment can introduce a row in the properties table if the default assigned to the user does not match the forced variant. We find this acceptable for the QA testers use case as few users are expected. For the other use case we have, campaign event organizers force assigning a variant to a group of users, this is not so ideal, as a percentage of rows will be indeed created.

In T375900#10279724, @cjming wrote:

• I opted to read a query parameter over a JS utility -- do we need to provide both?

The JS utility has come handy for us but I believe both methods are possible. What I wonder from the query parameter implementation is how to ensure the user has the variant permanently assigned, that means surviving across different pages, redirects, etc. I remember @phuedx brainstorming around client storage, cookies, etc in T374471#10141298, this has the problem of different experience in different browsers but maybe that's fine as an initial compromise solution.

• Do we need to provide custom segmentation for accounts by date or assign sampling rates by platform? This will increase complexity significantly for both MPIC and the MP extension - I'm wondering if this is in scope for this task.

From the GE point of view custom segmentation by account age is indeed valuable, in our next experiment we are targeting users since 2022-11-02 on (1081104). The sampling rates per platform were used in the past as a way of hiding a feature in a platform (eg: sample mobile 0), but that has been evaluated as not so relevant and could be done with a standard "feature". However, in many GE experiments, we use experiment sampling rates to incrementally rollout a feature so we can evaluate the impact on newcomers but also moderators burden. In general is seems a handy feature to be able to adapt the experiment population to the traffic of each wiki, per-platform, as these can differ, but it is not a hard requirement for Growth, @nettrom_WMF @Iflorez correct me if I'm wrong.

In T375900#10280284, @Sgs wrote:

In T375900#10279724, @cjming wrote:

• I opted to read a query parameter over a JS utility -- do we need to provide both?

The JS utility has come handy for us but I believe both methods are possible. What I wonder from the query parameter implementation is how to ensure the user has the variant permanently assigned, that means surviving across different pages, redirects, etc. I remember @phuedx brainstorming around client storage, cookies, etc in T374471#10141298, this has the problem of different experience in different browsers but maybe that's fine as an initial compromise solution.

In order to support the logged-out case, we're going to have to use a cookie 🍪. I think I'm OK with a querystring parameter as well but, as @Sgs said, it's difficult to assign and difficult to persist across pageviews. I think the querystring parameter should override the cookie?

Change #1084935 merged by jenkins-bot:

[mediawiki/extensions/MetricsPlatform@master] Allow users to override experiment enrollment

https://gerrit.wikimedia.org/r/1084935

Change #1134261 had a related patch set uploaded (by Umherirrender; author: Clare Ming):

[mediawiki/extensions/MetricsPlatform@REL1_43] Allow users to override experiment enrollment

https://gerrit.wikimedia.org/r/1134261