Background

Not all everyone experiments require splitting the cache. So-called "non-first-paint features" – features where the UX is client-side-only – could be experimented on without splitting the cache. For example:

• An experiment that varies the number of items in the search autocomplete
• An experiment that varies whether descriptions are shown in the search autocomplete
• An experiment that varies the delay before a Page Preview appears
• …

Splitting the cache for these experiments wouldn't affect the experiment and would place an unnecessary burden on the edge caches and MediaWiki. We foresaw this some time ago and decided that, for this class of experiment, Varnish would continue to enrol users but send the enrolment information to the browser via a Server-Timing header instead of to MediaWiki.

This epic covers updating the JS xLab SDK to reconcile the information received from MediaWiki and from Varnish via the Server-Timing header.

Scenarios

1. Cache-Splitting Experiment

Consider the experiment "A" which has a "control" group and a "treatment" group and splits the cache. When the user is enrolled:

• Varnish will split on a=treatment;
• We should expect the following values:

2. Non-Cache-Splitting Experiment

Consider the experiment "B" which has a "control" group and a "treatment" group and does not split the cache. When the user is enrolled:

• Varnish will not split the cache
• We should expect the following values:

(1) & (2) Concurrently

Consider the A & B experiments running concurrently. When the user is enrolled:

• Varnish will split the cache on a=treatment
• We should expect the following values:

---

This epic covers the implementation of non-cache splitting experiments in Test Kitchen.