Background
The Wikimedia Foundation will implement a first-party cookie for all traffic to our sites named WMF-Uniq. This cookie's implementation is designed to support future A/B testing, DDoS attack mitigation, and visitor trend analysis goals while preserving our users' privacy.
The identifiers in these cookies will only be processed in realtime at our private CDN edge servers. They are designed to contain the minimal amount of metadata necessary to meet functional needs, while meeting or exceeding the privacy requirements outlined in our privacy policy and cookie use guidelines. They will never be stored in traffic logs or databases, and we will not create user profiles of individual readers' browsing histories or patterns that could be linked to a specific person over time. All implementation details will be in public, open-source code. Users can choose to block or clear these cookies without any functional effect on their use of the sites.
KR/Hypothesis from Annual Planning
- SDS Objective 2: Product managers can quickly, easily, and confidently evaluate the impacts of product features.
- Key Result 2.4: By end of Q4 FY24/25, successfully enable one product team to run an A/B test on anonymous users for a first paint feature while maintaining privacy compliance and data integrity.
- Hypothesis 2.4.4: Traffic implementation of edge unique cookie (basically, this task)
Success Metrics
- Meet or exceed our privacy policy and cookie use guidelines
- Meet our timeline to use this cookie to complete one A/B test
Requirements
All items are implied to be done according to the design
- Exchange WMF-Uniq cookies with our users’ agents.
- Load A/B experiment configurations from a JSON file [which is distributed to CDN nodes initially by puppet]
- Identify requests which are from agents whose cookies are enrolled in configured experiments. For such requests:
- Send a Server-Timing header in the response to inform client-side Javascript of the experiment and group names.
- If the test varies caching (flag in experiment definition):
- Attach a header to the request before forwarding to the application layer (e.g. MediaWiki) which names the experiment and the group within the experiment, for application layer output-variance purposes.
- Split CDN caching for experimental groups based on the same experiment/group names in the header above.
- If the request is an A/B test result metric sent by client-side Javascript to our Eventgate analytics endpoint URI:
- In addition to the experiment/group name information above, attach a test-group-specific, derived, temporary agent ID to the header as well, which is constructed from a one-way hash of the user's raw cookie identifier and the metadata about the experiment itself.
Actual Task Work for Implementers
- Cut and package a first release version of the wmfuniq VMOD code. (T392059)
- Deploy the wmfuniq VMOD code to our production varnish servers.
- Deploy puppet patches to manage and distribute the server-side secret keys used to validate the cookies' contents.
- Deploy puppet patches to distribute a static JSON configuration file containing active A/B test configurations (initially empty).
- Deploy a VCL patch which makes all of our global Varnish servers minimally aware of the WMF-Uniq cookie and able to handle it by discarding and ignoring it.
- Drop X-Enrollment-Experiments in HAProxy to prevent UAs from polluting the CDN cache
- Avoid Vary: X-Enrollment-Experiments reaching the users
- Start varying the cache on X-Enrollment-Experiments
- Allow /evt-103e/v2/events to reach origin servers and send the traffic to intake-analytics
- Deploy a VCL patch which uses the VMOD to set and process these cookies correctly, which can be rolled out progressively to the CDN fleet by-edge-host, replacing the above minimal VCL.
- Progressively roll out to cache nodes and monitor for any functional, site performance, and/or server-load issues which may arise and address them, rolling back if/as necessary.
Timeline
The original timeline for rollout was to be complete by EOQ3 FY24-25 (end of March 2025). This schedule has already slipped, but we're aiming to begin the rollout of the full VCL patch for cookie processing during the week of April 21st. The other, enabling task items before that should be completed earlier. We're aiming to have the whole system functioning in time to conduct the first real A/B test by EOQ4 (end of June 2025).