Page MenuHomePhabricator

xLab: Include experiment enrollment in the PHP client library
Closed, ResolvedPublic5 Estimated Story Points

Description

T370880: [EPIC] FY 24/25 SDS 2.1.7 | Alpha Release of Instrument Configuration System (MPIC)

Description

This task is the PHP client update AC of T368326: Update Metrics Platform Client Libraries to accept experiment membership. Update the Metrics Platform PHP client library to include the experiments enrollment data object in the final submit method of the metrics client.

With the proposed updates to the experiment membership tracking schema (fragment proposed in T389995), we will need to update the PHP client library to supplements events i.e. be able to read and populate fields for the new experiment data object (to supercede the previously defined experiments data object)

Proposed payload to be added to all events:

experiment: {
  enrolled: "2025-hoverlinks-blue-3",
  assigned: "a_group_name",
  subject_id: "0000001badb00200000ff1ce1badb002000000ff1ce0000000ff1ce0000000",
  sampling_unit: "mw-user",
  other_assigned: {
    "experiment_name_2": "control",
    "experiment_name_3": "any_group"
  }
}

Acceptance Criteria

  • Update the prod PHP instruments to use the new web base schema
  • Read experiment details from wgMetricsPlatformUserExperiments and prepare them according to the experiment schema fragment
  • Supplement events with the experiment configuration details when it’s a requested contextual attribute
  • Decorate events with enrolled, assigned, subject_id, sampling_unit keys for experiment-related events, and other_assigned for all events.
  • Implement an Instrument class (similar to the one we have for the JS client library)
  • Leave enrollment details as they are when coordinator is previously set to custom. sampling_unit, subject_ids and other_assigned won't be filled and we will assume that enrolled and assigned were filled by the custom coordinator
  • coordinator will be set to forced when the assigned group for the enrolled experiment has been overriden

Required

  • Unit/Integration tests?
  • Documentation?
  • Passed QA?

Details

Related Changes in Gerrit:
Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
Add Instrument class to PHP libraryrepos/data-engineering/metrics-platform!85cjmingT374840-add-experiment-enrollment-php-librarymain
Customize query in GitLab

Event Timeline

Sfaci updated the task description. (Show Details)
Sfaci updated the task description. (Show Details)
Milimetric triaged this task as Medium priority.Nov 12 2024, 4:00 PM
cjming renamed this task from Update Metrics Platform PHP library to include experiment enrollment to xLab: Include experiment enrollment in the PHP client library.Mar 26 2025, 4:32 PM
cjming claimed this task.
cjming updated the task description. (Show Details)

Change #1134668 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[mediawiki/extensions/MetricsPlatform@master] Update experiment manager for PHP

https://gerrit.wikimedia.org/r/1134668

Change #1134669 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[mediawiki/extensions/EventLogging@master] Include experiment enrollment to PHP

https://gerrit.wikimedia.org/r/1134669

With test code in a hook and associated patches:

$interactionData = [
	'action_source' => 'value for action source',
	'action_context' => 'value for action context'
];
$streamName = 'product_metrics.web_base';
$action = 'hover';
$schemaID = '/analytics/product_metrics/web/base/1.4.1';

EventLogging::getMetricsPlatformClient( $experimentManager->getExperimentEnrollments() )->submitInteraction(
	$streamName,
	$schemaID,
	$action,
	$interactionData,
	'experimentation-lab-test-1-experiment'
);

I'm able to see events decorated with the experiment data object in my debugger:

Screenshot 2025-04-07 at 5.33.30 AM.png (1×3 px, 868 KB)

To Do:

  • troubleshoot event validation in local eventgate
  • address a few more ACs
  • add tests

woohoo - got local eventgate working:

Screenshot 2025-04-07 at 3.26.58 PM.png (1×2 px, 591 KB)

now working on tests and remaining ACs

Testing Instructions

Check out the following MR/patches:

Include the following snippet to a hook - i.e. \MetricsPlatform\includes\Hooks.php

// Inside onBeforePageDisplay()

EventLogging::getMetricsPlatformClient( $experimentManager->getExperimentEnrollments() )->submitInteraction(
	'product_metrics.web_base',
	'/analytics/product_metrics/web/base/1.4.1',
	'hover',
	[
		'action_source' => 'the action source',
		'action_context' => 'the action context'
	],
	'experimentation-lab-test-1-experiment'
);

Open a local MW instance as a logged-in user and note the the validated event in local EventGate:

Screenshot 2025-04-08 at 10.38.52 PM.png (1×2 px, 413 KB)

Milimetric set the point value for this task to 5.Apr 10 2025, 2:04 PM

Change #1134669 abandoned by Clare Ming:

[mediawiki/extensions/EventLogging@master] Include experiment enrollment to PHP

Reason:

not relevant

https://gerrit.wikimedia.org/r/1134669

Latest Testing Instructions

Check out the following patch:

Configure experiments in xLab:

  • create and activate a new experiment named experimentation-lab-test-1-experiment in your local xLab (or grab the slug for an active experiment in your local xLab instance)
  • create and activate a few more experiments (to test other_assigned key in the experiment data object) and note their slugs as well
  • to ensure you are enrolled, set sampling rates to 1 for all activated experiments

Include the following snippet to the Metrics Platform onBeforePageDisplay hook - i.e. \MetricsPlatform\includes\Hooks.php after L142 (when the current user is enrolled and experiment config is set in the ExperimentManager)

  • Replace the parameter in ExperimentManager::getExperiment() with the slug of an experiment that is active in your xLab instance
$experiment = $experimentManager->getExperiment( 'experimentation-lab-test-1-experiment' );

$experiment->send(
    'validate experiment details',
    [
        'action_source' => 'Metrics Platform onBeforePageDisplay hook',
        'action_context' => 'experiment enrollment'
    ]
);

Open a local MW instance as a logged-in user and note the the validated event in local EventGate:

Screenshot 2025-04-11 at 5.25.54 PM.png (1×2 px, 316 KB)

Change #1134668 merged by jenkins-bot:

[mediawiki/extensions/MetricsPlatform@master] Update PHP experiment manager for experiment enrollment

https://gerrit.wikimedia.org/r/1134668

Just a note for posterity -- it turns out that most of the work for getting experiment enrollment details in events is happening/ has happened in the Metrics Platform extension in the SDKs rather than in the client libraries themselves.

Per our discussion earlier today, we originally thought updates would be needed in the client libs to accommodate for experiments but the abstractions of the submit methods in the libraries are flexible enough to be used as is by the SDKs in the Metrics Platform extension and by the Experiment Manager therein. As such, I plan to close the hypothesis 2.4.7 about the client libraries tomorrow.