Background
Growth team is working on the adoption of xLab for experimentation. The feature we want to experiment with is Leveling up notifications. These notifications are sent to users 48h after they register with some engaging messages that vary according to the number of edits the user has done by that time.
In order to send these notifications, GrowthExperiments uses the LocalUserCreated hook to schedule a job with a release timestamp of +48h (see source). GrowthExperiments needs to know if the user is enrolled in the experiment and in which group by this time so it can decide to send different messaging for group A and group B or even send an extra notification for the treatment group.
The only point of enrollment xLab has built in now is BeforeInitialize, which is called early in the request lifecycle. Earlier than LocalUserCreated in an authentication flow. By the time xLab requests enrollment the user object in context is non-registered, so xLab skips enrollment for active logged in experiments (see source), returning a null assignment.
In order to circumvent this problem GrowthExperiments is requesting re-enrollment to xLab before scheduling its notifications so XLab's experiment manager knows about the logged in experiments and is able to enroll users into the experiment and assign some group. To do so, we're re-doing the same steps xLab does with a guaranteed registered user:
- Fetch configs
- Enroll user
- Initialize manager with enrollment result
- Decorate output with xLab client JS config vars and body CSS classes
Feature summary (what you would like to be able to do and where):
Ideally GrowthExperiments should do nothing for getting a user enrolled in logged in experiments on account creation. However if that's a non standard use-case it would be sufficient that xLab provides a mechanism to allow re-enrollment at specific places.
Another use cases for Growth that has come up is the retrieval of an assigned group within an action API or rest request.
Use case(s) (list the steps that you performed to discover that problem, and describe the actual underlying problem which you want to solve. Do not describe only a solution):
- Experiment assigned group retrieval on account creation
- Experiment assigned group retrieval on API calls
- Experiment assigned group retrieval on maintenance scripts
- ...
Benefits (why should this be implemented?):
Adoption of xLab for more complex features that do not only change web UI but more broad behaviors of the system.
Questions
- The retrieval of a central ID right after account creation (LocalUserCreated) has been problematic in the past. Are there any better alternatives to subscribe to an account creation and ensure relevant data is there, even at the cost of some delay? Would Domain events be suitable for this?
- What points of enrollment should xLab support? BeforeInitialize seems only covering a subset of valid use cases, specially for logged in experiments.
