Background

With the new experimentation platform, there is a new method of doing instrumentation that is (largely) much easier! In addition to that, we are only instrumenting clickthrough rate

This ticket encompasses starting the clickthrough rate instrument, which has already been written, and passing in the experiment object

Refer to the proof of concept in T395944 - note that for local development, you may need to add the following to your stream config:

'product_metrics.web_base' => [
	'schema_title' => 'analytics/product_metrics/web/base',
	'destination_event_service' => 'eventgate-analytics-external',
	'producers' => [
		'metrics_platform_client' => [
			'provide_values' => [
				'mediawiki_database',
				'page_id',
				'page_title',
				'page_namespace_id',
				'mediawiki_skin',
				'page_content_language',
				'agent_client_platform',
				'agent_client_platform_family',
				'performer_session_id',
				'performer_name',
				'performer_is_bot',
				'performer_is_logged_in',
				'performer_is_temp',
			],
		],
	],
	'sample' => [
		'unit' => 'pageview',
		'rate' => 1,
	],
]

Further documentation

https://wikitech.wikimedia.org/wiki/Experimentation_Lab/Measuring_clickthrough_rates

User story

As an analyst supporting the donate A/B test hypothesis, I would like to be able to look at events fired pertaining to clickthrough rate for the experiment

Requirements

• the clickthrough instrument is started
  • the instrument targets the selector added in T395861
  • an event is fired upon impression (in this case page load)
  • an event is fired when the control link is clicked
  • an event is fired when the treatment button is clicked
  • in all cases, the event contains an experiment fragment that has group info present
• no events are fired in the not-in-experiment case

Requirements (prior to Experiment activation)

Ensure that no product_metrics.web_base events related to the donate link experiment are being fired in either the beta or production environments, as the experiment is currently turned off. This includes:
• No events on page impression
• No events on control or treatment link/button click

BDD (prior to Experiment activation)

Feature: Donate link experiment instrumentation is inactive

  Scenario: No instrumentation when experiment is off
    Given I visit a page in Vector skin in beta or production
    When the page loads
    Then no product_metrics.web_base events are fired

  Scenario: No clickthrough event for control donate link
    Given I click the control donate link in beta or production
    Then no product_metrics.web_base events are fired

Test Steps (prior to Experiment activation)

Test Case 1: Verify no instrumentation on page load

1. Open browser dev tools → Network tab → filter for eventgate-analytics.
2. Navigate to https://en.wikipedia.beta.wmflabs.org/wiki/Main_Page?useskin=vector.
3. AC1: Verify that no product_metrics.web_base events are fired on page load.

Test Case 2: Verify no instrumentation on control donate link click

1. Ensure the control donate link is visible on the page.
2. Click the control donate link.
3. Observe the Network tab.
4. AC2: Verify that no product_metrics.web_base events are fired on link click.

QA Results - Beta

QA Results - Prod

BDD ( with Experiment activated )

Feature: Donate clickthrough instrumentation

  Scenario: Instrument starts and fires impression event
    Given I visit a Vector page while assigned to the experiment
    When the page loads
    Then the clickthrough instrument is started
    And the instrument targets the donate element defined by the selector from T395861
    And an impression event is fired
    And the event contains an experiment fragment with my group

  Scenario: Control link click fires event
    Given I am assigned to the experiment control variant
    When I click the control donate link
    Then a click event is fired
    And the event contains an experiment fragment with my group

  Scenario: Treatment button click fires event
    Given I am assigned to the experiment treatment variant
    When I click the treatment donate button
    Then a click event is fired
    And the event contains an experiment fragment with my group

  Scenario: No events when not in experiment
    Given I am not in the experiment
    When I load the page and click the donate UI
    Then no clickthrough instrumentation events are fired

Test Steps ( with Experiment activated )

Test Steps

Test Case 1: Instrument starts, targets selector, and fires impression

1. Assign yourself to the experiment (any variant), e.g. append ?mpo=we-3-2-3-donate-ab-test-1:treatment (or :control) to a Vector page URL.
2. Load the page.
3. AC1: Verify the clickthrough instrument is started (per the team’s standard inspection method).
4. AC2: Verify the donate UI element that matches the selector from T395861 is targeted by the instrument (e.g., appropriate handlers are bound).
5. AC3: Verify an impression event is fired and includes an experiment fragment with the correct group.

Test Case 2: Control click event

1. Assign control variant: append ?mpo=we-3-2-3-donate-ab-test-1:control to the page URL.
2. Click the control donate link (the element defined for control in T395861).
3. AC4: Verify a click event is fired.
4. AC5: Verify the event includes an experiment fragment with group=control.

Test Case 3: Treatment click event

1. Assign treatment variant: append ?mpo=we-3-2-3-donate-ab-test-1:treatment to the page URL.
2. Click the treatment donate button (the element defined for treatment in T395861).
3. AC6: Verify a click event is fired.
4. AC7: Verify the event includes an experiment fragment with group=treatment.

Test Case 4: Not-in-experiment produces no events

1. Load the same page without any experiment assignment parameter (no mpo).
2. Interact with the donate UI (link/button if present).
3. AC8: Verify no impression event from this instrumentation is fired on page load.
4. AC9: Verify no click event from this instrumentation is fired on interaction.

QA Results - Beta

Design

N/A

Acceptance criteria

We are able to verify that events have been fired for both impression and click, with information about which group the user was in present

Communication criteria - does this need an announcement or discussion?

N/A

Rollback plan

This should be easily revertible as it's all javascript. If we mess up the data we may need to use a separate label or experiment, but it should be okay