Page MenuHomePhabricator

Run synthetic experiment using PHP SDK in Test Kitchen
Open, HighPublic1 Estimated Story Points

Description

Background

To verify that the switch to Test Kitchen is working as expected (see T407806: Rename Metrics Platform Extension to Test Kitchen), run an experiment using the PHP SDK in Test Kitchen.

Description

Replicate previous experiment to ensure the Test Kitchen PHP SDK is working properly.

Previous experiment:

Acceptance Criteria

  • Expected results are confirmed in Hive
  • Instrumentation is removed upon verification

Event Timeline

cjming renamed this task from Run synthetic experiment using PHP SDK with Test Kitchen to Run synthetic experiment using PHP SDK in Test Kitchen.Jan 14 2026, 7:35 AM

Change #1227426 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[mediawiki/extensions/TestKitchen@master] Update experiment code for JS, PHP SDKs testing of TK

https://gerrit.wikimedia.org/r/1227426

Change #1227426 merged by jenkins-bot:

[mediawiki/extensions/TestKitchen@master] Update experiment code for JS, PHP SDKs testing of TK

https://gerrit.wikimedia.org/r/1227426

Change #1227435 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[mediawiki/extensions/TestKitchen@wmf/1.46.0-wmf.11] Update experiment code for JS, PHP SDKs testing of TK

https://gerrit.wikimedia.org/r/1227435

Change #1227435 merged by jenkins-bot:

[mediawiki/extensions/TestKitchen@wmf/1.46.0-wmf.11] Update experiment code for JS, PHP SDKs testing of TK

https://gerrit.wikimedia.org/r/1227435

Mentioned in SAL (#wikimedia-operations) [2026-01-15T21:53:05Z] <cjming@deploy2002> Started scap sync-world: Backport for [[gerrit:1227435|Update experiment code for JS, PHP SDKs testing of TK (T414528 T414530)]]

Mentioned in SAL (#wikimedia-operations) [2026-01-15T21:55:07Z] <cjming@deploy2002> cjming: Backport for [[gerrit:1227435|Update experiment code for JS, PHP SDKs testing of TK (T414528 T414530)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-01-15T22:02:29Z] <cjming@deploy2002> Finished scap sync-world: Backport for [[gerrit:1227435|Update experiment code for JS, PHP SDKs testing of TK (T414528 T414530)]] (duration: 09m 23s)

cjming triaged this task as High priority.
cjming set the point value for this task to 2.

Process for testing PHP SDK using Test Kitchen extension:

  1. PHP experiment was configured and initiated yesterday on Japanese Wikipedia at 10% sampling
  2. Experiment code for both SDKs was backported to 1.46.0-wmf.11 yesterday
  3. Verified via Kafka topic streaming and running queries on analytics servers:

Confirmed that events were streaming in via:

kafkacat -C -b kafka-jumbo1010.eqiad.wmnet:9092 -t eqiad.product_metrics.web_base -o -100000 | grep PageVisitTestKitchen

Screenshot 2026-01-15 at 3.05.20 PM.png (856×2 px, 646 KB)

Ran the following queries:

SELECT count(*)
FROM event.product_metrics_web_base
WHERE instrument_name = 'PageVisitTestKitchen'
AND year = 2026 AND month = 01 AND day = 16;

// Returns 27377

SELECT count(*)
FROM event.product_metrics_web_base
WHERE instrument_name = 'PageVisitTestKitchen'
AND experiment.assigned='control'
AND year = 2026 AND month = 01 AND day = 16;

// Returns 13052

SELECT count(*)
FROM event.product_metrics_web_base
WHERE instrument_name = 'PageVisitTestKitchen'
AND experiment.assigned='control-2'
AND year = 2026 AND month = 01 AND day = 16;

// Returns 14325

With confirmations above, I will turn off experiment and remove related code.

Change #1227902 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[mediawiki/extensions/TestKitchen@master] Remove experiment code testing TK SDKs

https://gerrit.wikimedia.org/r/1227902

Change #1227902 merged by jenkins-bot:

[mediawiki/extensions/TestKitchen@master] Remove experiment code testing TK SDKs

https://gerrit.wikimedia.org/r/1227902