While discussing this with @phuedx and @KReid-WMF, it wasn't clear whether this is the right way to go and that motivating examples / use cases would help us arrive at the best solution to the underlying problem.

Let's imagine some experiments that use edge uniques for enrollment and use the PHP SDK to render different versions of pages for enrolled users:

• Different design of or text copy on Special:CreateAccount
  • Exposure event only when visiting the page
• Bringing one of the links under Tools dropdown/sidebar menu out (say, Permanent link), to sit beside Read, Edit, and View history links
  • We can't just check for presence of Permanent link to log exposure, because we want to log exposure for control group too, whose Permanent link would still be hidden under Tools.
• Adding page stats (# of editors who edited page, # of edits, how long the page has existed, pageviews in last 30 days) to the top of the article
  • Feature only on Article pages
  • Similar to other example, exposure for control group = lack of feature, while exposure for treatment group = presence of feature

For these examples, maybe we could consider how difficult it would be for developers to instrument their experiment's exposure on the client-side such that there are no false positives (logging exposure when there was no exposure) or false negatives (not logging exposure when there was exposure).

Additional scenarios where the concept of exposure is tricky:

• Different formatting of the references section on articles that have references (Parsoid change?)
  • Here we would want to discourage logging exposure server-side because when user visits an article with references, the references section might not be visible right away. So any kind of exposure event would need to be deferred until the references section comes into view, similar to CTR instrumentation spec where we want to log impression of UI elements only when they're actually visible.

Hm… HMMMM…

I wonder if it would be possible to let developers put a div.test-kitchen-enrolled-{experiment_name}-assigned-{group_name} (or something) around their experiment content, even if the user is a control group and nothing new is being rendered. And then we have an instrument that automatically logs exposure when a div with that class becomes visible. So if I'm a developer, I have PHP code that stuffs my change (or lack of change!) into a div and that's basically how I log exposure from server-side.

I don't think this goes against the decision that exposure logging should be intentional (we shouldn't automatically log exposure), this just creates a mechanism through which one still intentionally logs exposure, just without calling Experiment#logExposure().

Or maybe just inserting an empty div (with experiment ID and variation ID embedded into class or as data attributes) whenever we want to indicate exposure from server-side?

@KReid-WMF asked

Different design of or text copy on Special:CreateAccount

I think I see why we can't do this from js - because there's no js code change associated with the experiment. Is that right? If so, why not make it a requirement to add a js listener to page load that sends an event instead of asking the server to tell the js?

Great question! So in this particular case I think we could have a tightly scoped client-side instrument for logging exposure:

const specialPageName = mw.config.get( 'wgCanonicalSpecialPageName' );
if ( specialPageName === 'CreateAccount' ) {
	mw.loader.using( 'ext.testKitchen' ).then( () => {
		const experiment = mw.testKitchen.getExperiment( 'my-account-creation-experiment' );
		experiment.logExposure();
	} );
}

which would run on every page load but only log exposure on visits (pending T414738: Minimize volume of exposure events sent) to Special:CreateAccount and do nothing on visits to all other pages.

This example benefits from existence of mw.config.get( 'wgCanonicalSpecialPageName' ), sure, but that's not a general solution.

Updated task description and AC to be about desired outcome, rather than the solution since we're not sure yet what the solution should be.

Or maybe just inserting an empty div (with experiment ID and variation ID embedded into class or as data attributes) whenever we want to indicate exposure from server-side?

We could totally attach a visibility checker to a <div data-experiment-id="my-exeriment" data-variation-id="control"></div> and log exposure based on experiment ID and variation ID in those attributes.

EDIT: Technically we don't even need variation ID. Just experiment ID would do. Regardless of what other event logging is happening on the page from the developer's instrumentation, we could detect the presence of this div and then for every such div present on the page we extract the experiment ID and do mw.testKitchen.getExperiment( experimentId ).logExposure();

The div suggestion is interesting - I agree that it doesn't violate the spirit of exposure logging should be intentional, but am uncertain which of the possible approaches will be least complex code-wise.

Do we have people waiting to run experiments with these cases? And do we know how hard the div case would be to build for?

In the spirit of keeping to the smallest unit of work that provides value for users at a time, if it's complex to build a solution for these cases, I might suggest we hold off on building it until we have users ready to run something. That's an easier call to make if it's possible for them to do a more complex js thing if they're in a hurry than if we have to tell them to wait for it to be built to run at all.

Do we have people waiting to run experiments with these cases?

David from Editing shared that their experiments use client-side changes exclusively, so they wouldn't be doing changes server-side and needing to log exposure client-side.

Marco from Reader Growth shared their server-side implementation with me. In summary:

• The code makes use of a helper function isInAnyTreatmentGroup which checks if the request is assigned to any group for a given experiment
• If the request is assigned to any group and satisfies other conditions (e.g. main namespace, skin), it loads ext.readerExperiments.imageBrowsing module and also adds <div id="ext-readerExperiments-imageBrowsing"></div> to the response.
• Therefore, all users enrolled in the experiment would have this client-side module loaded.
• The module then checks for enrollment/assignment and
  • If user is in control group, do nothing to div#ext-readerExperiments-imageBrowsing
  • If user is in treatment group, populate div#ext-readerExperiments-imageBrowsing

With this pattern, they could easily call Experiment#logExposure() on the client-side and have it be accurate (only log exposure when there is actually exposure to any variation).

So they effectively have a solution to this task simply by only loading & running client-side code using server-side logic. As long as they continue to use this pattern for implementing their experiments, they're all good.

And do we know how hard the div case would be to build for?

Sam would be a far better judge of this than I am, but looking at their use of prependHTML() in the aforementioned implementation to insert an empty div which would be populated client-side for users in the treatment group:

private function maybeInitImageBrowsing( OutputPage $out ): void {
	$context = $out->getContext();
	$request = $context->getRequest();
	$title = $context->getTitle();

	// Enable if Minerva skin AND (URL param is set OR user is in any experiment's treatment group).
	if (
		$title && $title->getNamespace() === NS_MAIN &&
		$out->getSkin()->getSkinName() === 'minerva' &&
		(
			$this->isInAnyTreatmentGroup( $request, self::IMAGE_BROWSING_EXPERIMENTS ) ||
			$request->getFuzzyBool( 'imageBrowsing' )
		)
	) {
		$out->prependHTML(
			'<div id="ext-readerExperiments-imageBrowsing"></div>'
		);

		$out->addModuleStyles( 'ext.readerExperiments.imageBrowsing.styles' );

		// Load heavy module since already gated server-side.
		$out->addModules( 'ext.readerExperiments.imageBrowsing' );
	}
}

Doesn't seem particularly hard?

@KReid-WMF @phuedx: I think Reader Growth has a pretty good pattern/practice that we can recommend in the docs as part of T414735: Document guidelines, scenarios, and recommendations on exposure logging.

So I propose we decline this task for now and keep the div idea in our pockets in case a need arises and we reopen this task.

Some thoughts here:

1. We already add classes to the <body> element when the user has been enrolled in an experiment. This is already a signal from the server
2. Adding additional markup can break existing styles. Developers will be aware of this though
3. As I mentioned elsewhere, there's a lot of nuance around RL modules and their cost. Defining a new RL module adds a cost to all pageviews. RL modules and are minified in the context of the other modules that are being loaded at the same time, so sometimes it's more performant to include a new file in an old module (see the ext.wikimediaEvents RL module, which holds a number of experiment-related instruments)
4. There isn't going to be a one-size-fits-all solution. The general guidance of "Only log exposure when the user is actually exposed to the experiment" with a library of examples might have to be good enough
5. There are some cases where we could automate this, e.g. with special markup, but I'm not convinced that this will be a good developer experience for anything other than a simple experiment. There will come a time when enough experiments have been run that we can extract patterns from their implementations but we're not there yet