===Scope===
- What would generalized enrolment authority look like?
- What are its enrolment authority in relation to a different enrolment authority?
- What would an API look like for enrolment on a custom token?
- What are the responsibilities and what should it do?
- Is non cache splitting just another enrolment authority?
===Obvious Gotchas===
- If Test Kitchen enrols people automatically on global identifier, what does it mean to enrol based on local identifier? How do we represent that?
- Don't be influenced by what's currently written - what is the ideal state and how do we move backwards from that?
- Conceptual doozie
=== Outcome
==== What do they do?
1. An experiment enrollment sampling authority ensures that all group assignments are equally likely for all experiments
1. For example, if there are two experiments running, both with control and treatment groups, then the user should be enrolled into the experiments in such a way that the following experimental group assignments are equally likely:
| Experiment 1 | Experiment 2 |
| ------------ | ------------ |
| control | control |
| control | treatment |
| treatment | control |
| treatment | treatment |
2. An EESA consistently assigns the same user to the same experimental group for the same experiment
==== What don't they do?
1. Manage the identifiers that represent the users
2. Coordinate the enrollment of a user into multiple experiments, which may or may not require different identifiers
3. Decorate the output with metadata about the enrollment of the user into experiments
==== Example 1: [T405074 xLab: Allow user re-enrollment at specific times](https://phabricator.wikimedia.org/T405074)
Test Kitchen enrolls the user in experiments that require the `mw-user` identifier type in the `BeforeInitialize` hook using the central user ID. Unfortunately, the Growth team discovered that the central user ID isn't available to `BeforeInitialize` hook handlers during the account creation flow. In order to work around this limitation, the Growth team [reimplemented the enrollment coordination and output decoration steps](https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/GrowthExperiments/+/master/includes/ExperimentXLabManager.php#48). They had to do this because Test Kitchen doesn't separate EESAs from the enrollment sampling coordination and output decoration steps.
---
**Recommendation #1: Defer Output Decoration**
Currently, the output decoration step is performed immediately after the experiment enrollment sampling step in the `BeforeInitialize` hook handler. It could and should be performed much later in the `BeforePageDisplay` hook handler. This is consistent with (4) in § What don't they do? above.
---
**Recommendation #2: Allow Identifiers to be Updated**
In order to handle the rare case where an identifier type isn't available, the experiment enrollment coordinator should allow identifiers to be updated. Unfortunately, hook handlers are called in the order in which they are registered so we should provide an API.
**PHP**
```php
namespace MediaWiki\Extension\MetricsPlatform\TestKitchen;
interface Coordinator {
const IDENTIFIER_TYPE_EDGE_UNIQUE = 'edge-unique';
const IDENTIFIER_TYPE_MW_USER = 'mw-user';
/**
* Refreshes the identifier.
*
* When an identifier is refreshed all experiments that require the
* identifier type are enrolled and the `XLab.ExperimentManager` service is
* updated with those enrollments. If this is done before the
* BeforePageOutput` hook is run, then the output is decorated with those
* enrollments also.
*
* @throws \DomainException If the identifier type can't be refreshed, i.e.
* it's {@link Coordinator::IDENTIFIER_TYPE_MW_USER}
*/
public function refreshIdentifier( string $identifierType, string $identifier );
}
```
**JavaScript**
```typescript
namespace mw {
namespace testKitchen {
enum IdentifierType {
EDGE_UNIQUE = 'edge-unique',
MW_USER = 'mw-user'
}
interface Coordinator {
/**
* Refreshes the identifier.
*
* When an identifier is refreshed all experiments that require the
* identifier type are enrolled and the body is decorated with CSS
* classes corresponding to those enrollments.
*
* @throws Error If the identifier type can't be refreshed, i.e.
* it's {@link IdentifierType.EDGE_UNIQUE} or
* {@link IDENTIFIER_TYPE.MW_USER}
*/
refreshIdentifier(
identifierType: IdentifierType,
identifier: string
);
}
}
}
```
---
**Recommendation #3: Generalize Output Decoration**
It's conceivable that Test Kitchen could decorate the output of the MediaWiki Action and REST APIs. Test Kitchen should select and use the appropriate output decorator.
---
Quoting T405074:
>Another use cases for Growth that has come up is the retrieval of an assigned group within an action API or rest request.
>
>**Use case(s)** (list the steps that you performed to discover that problem, and describe the actual underlying problem which you want to solve. Do not describe only a solution):
>
>- Experiment assigned group retrieval on account creation
>- Experiment assigned group retrieval on API calls
>- Experiment assigned group retrieval on maintenance scripts
Uses cases (1) and (2) are covered by recommendations (1), (2), and (3) above. However, those recommendations assume that the MediaWiki application is responding to an HTTP request, which doesn't hold in use case (3). Maintenance scripts don't have `BeforeInitialize` and `LocalUserCreated` hook handlers, they don't have output to decorate, and, crucially, they are userless.
---
**Recommendation #4: Expose Experiment Enrollment Sampling**
```php
namespace MediaWiki\Extension\MetricsPlatform\TestKitchen;
interface Coordinator {
/**
* Gets the experimental group assignment for the identifier.
*
* This method is pure – it has no side effects.
*
* @return {string|null} If the user is enrolled in the experiment, then
* then the name of the experimental group; otherwise, `null`
*/
public function getEnrollmentForIdentifier( string $experimentName, string $identifier ): ?string;
}
```