In order to facilitate the count of each group in the experiment, it's been requested to have a single stream to collect the information user_id-user_variant. Since GrowthExperiments does not have any instrument in place that collects such information (homepagemodule was used priorily for this), it's necessary to add it to some existing instrument or create a dedicated one.
Possible solutions for new accounts
For the surfacing experiment targeting new accounts, we'd want to track variant assigned at the time of account creation (similar to what we do with statslib metrics in our Growth team kpi dashboard, account creation section).
- [Discarded] Make use of the already approved and running serversideaccountcreation stream which can track a campaign parameter (see CampaignsSecondaryAuthenticationProvider.php#L79-L90) and use the user variant assigned as the campaign value, which should be later easy to use for counting new accounts on each group.
The variant could be added from onAuthFormFieldsChange hook- Discarded because how to feed such parameter "during" account creating is not clear, onAuthFormFieldsChange is not an option since it is too early in the process, the user id is not been created yet.
- Create a new dedicated stream to track an "experiment enrollment event on account creation", increment the metric using LocalUserCreated hook, along with the condition local-bucket-growth for user options.
- Re-use an available "account creation" interaction logger, maybe from T346327, although I'm not sure yet how the integration between Growth and WE would work or if it's a good idea
Possible solutions for existing accounts
For the surfacing experiment targeting existing accounts, we'd want to track variant assigned at the time of viewing a page. Ideally a page that has recommendations, but this is left for discussion, nice to have.
- Track user id and assigned variant in the BeforePageDisplayHook execution as an impression, this has the benefits of not adding extra JS payload to the response of "control" users, but the downsides of potentially counting impressions of users that wouldn't be elegible for the experiment for product or system constraints. eg: JS didn't load, link recommendations API did not return any results
- Track user id and assigned variant in the client using LinkSuggestionInteractionLogger, this has the benefit that only users who would be able to load the feature and interact are counted. It has the downside of a bigger response payload. (Could be optimized with some lazy loading?)
Acceptance criteria
- Data analyst can query for counting the number of users enrolled on each experiment group on account creation
- [Nice to have] Data analyst can query for counting the number of users enrolled on each experiment group on page impression (including new accounts)