NOTE: {T414729} includes basic instructions on usage when conducting experiments, this task is about more extensive documentation
Once exposure logging is implemented, we should provide developers with more thorough guidance for logging exposure in their experiments to ensure accuracy and trustworthiness of the data collected.
## Notes
For everyone (all user traffic) experiments (that use edge uniques for enrollment) with server-side changes, we can borrow ideas from this approach:
>>! In T414732#11570245, @mpopov wrote:
> Marco from Reader Growth shared [[ https://github.com/wikimedia/mediawiki-extensions-ReaderExperiments/blob/043716c5a3050c702b552eb045c02d0cf9cb83c7/src/Hooks.php#L166 | their server-side implementation ]] with me. In summary:
> - The code makes use of a helper function [[ https://github.com/wikimedia/mediawiki-extensions-ReaderExperiments/blob/043716c5a3050c702b552eb045c02d0cf9cb83c7/src/Hooks.php#L66 | isInAnyTreatmentGroup ]] which checks if the request is assigned to any group for a given experiment
> - If the request is assigned to any group and satisfies other conditions (e.g. main namespace, skin), it loads `ext.readerExperiments.imageBrowsing` module and also adds `<div id="ext-readerExperiments-imageBrowsing"></div>` to the response.
> - Therefore, all users enrolled in the experiment would have this client-side module loaded.
> - The module then checks for enrollment/assignment and
> - If user is in control group, do nothing to `div#ext-readerExperiments-imageBrowsing`
> - If user is in treatment group, populate `div#ext-readerExperiments-imageBrowsing`
> With this pattern, they could easily call `Experiment#logExposure()` on the client-side and have it be accurate (only log exposure when there is actually exposure to any variation).
>
> So they effectively have a solution to this task simply by only loading & running client-side code using server-side logic. As long as they continue to use this pattern for implementing their experiments, they're all good.
We can provide a recommendation for how to log exposure on the client-side, namely by using server-side logic to conditionally load client-side instrumentation which would include an exposure logging call.
>>! In T414732#11582833, @phuedx wrote:
> Some thoughts here:
>
> 1. We already add classes to the `<body>` element when the user has been enrolled in an experiment. This is already a signal from the server
> 1. Adding additional markup can break existing styles. Developers will be aware of this though
> 1. As I mentioned elsewhere, there's a lot of nuance around RL modules and their cost. Defining a new RL module adds a cost to //all// pageviews. RL modules and are minified in the context of the other modules that are being loaded at the same time, so sometimes it's more performant to include a new file in an old module (see the `ext.wikimediaEvents` RL module, which holds a number of experiment-related instruments)
> 1. There isn't going to be a one-size-fits-all solution. The general guidance of "Only log exposure when the user is actually exposed to the experiment" with a library of examples might have to be good enough
> 1. There are some cases where we could automate this, e.g. with special markup, but I'm not convinced that this will be a good developer experience for anything other than a simple experiment. There will come a time when enough experiments have been run that we can extract patterns from their implementations but we're not there yet
## Acceptance criteria
[ ] https://wikitech.wikimedia.org/wiki/Test_Kitchen/Experiment_exposure_logging includes:
- examples/scenarios
- do's & don't's
- recommended patterns / best practices
[ ] https://wikitech.wikimedia.org/wiki/Test_Kitchen/Conduct_an_experiment links to https://wikitech.wikimedia.org/wiki/Test_Kitchen/Experiment_exposure_logging