I suspect the way to do this is to create a brand new extension called Test Kitchen and have that deployed with all the renamed bits, get it propagated, point all references to the new extension, and then decommission the old extension?

confirmed yes ^^

Requested new repo:
https://www.mediawiki.org/wiki/Gerrit/New_repositories/Requests#mediawiki/extensions/TestKitchen

If the Metrics Platform Extension (#metrics_platform = Metrics Platform) will be renamed to Test Kitchen, how not to collide with the existing #Test_Kitchen tag at https://phabricator.wikimedia.org/project/profile/6630/ ? What is supposed to get renamed in Phabricator? cf T407900

In T407806#11424659, @Aklapper wrote:

If the Metrics Platform Extension (#metrics_platform = Metrics Platform) will be renamed to Test Kitchen, how not to collide with the existing #Test_Kitchen tag at https://phabricator.wikimedia.org/project/profile/6630/ ? What is supposed to get renamed in Phabricator?

hi @Aklapper - i'm not sure what the fate of Metrics Platform should be - aiui, it's not an active board we use -- I already went ahead and renamed our project and workboard in Phab (previously Experimentation Lab) to Test Kitchen - https://phabricator.wikimedia.org/project/board/6630/

@JVanderhoop-WMF @Milimetric @KReid-WMF - should Metrics Platform be retired/removed?

I'm removing mentions (and making sure they land on the Test Kitchen board) - then we can delete the Metrics Platform board. Please hold.

Change #1214551 had a related patch set uploaded (by Reedy; author: Reedy):

[integration/config@master] zuul: Copy MetricsPlatform CI to TestKitchen

https://gerrit.wikimedia.org/r/1214551

Change #1214551 merged by jenkins-bot:

[integration/config@master] zuul: Copy MetricsPlatform CI to TestKitchen

https://gerrit.wikimedia.org/r/1214551

Mentioned in SAL (#wikimedia-releng) [2025-12-03T16:11:48Z] <Reedy> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1214551 T407806

I have now archived the Metrics Platform board

Change #1216705 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[mediawiki/extensions/TestKitchen@master] Initialize Test Kitchen with revised files from Metrics Platform

https://gerrit.wikimedia.org/r/1216705

cjming opened https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/226

make-release: Start branching TestKitchen

Change #1216847 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/mediawiki-config@master] extension-list: Add Test Kitchen

https://gerrit.wikimedia.org/r/1216847

Change #1217360 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/mediawiki-config@master] Deploy TestKitchen to Beta Cluster

https://gerrit.wikimedia.org/r/1217360

Since this is a simple extension rename, an application security review is not needed

awaiting branch cut from Release Engineering (they have been at an offsite this week) - posted to #developer-experience channel in Slack looking for someone to merge

thcipriani merged https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/226

make-release: Start branching TestKitchen

yay! so branch has been cut - leaving this in blocked for now until release branch has fully propagated

Change #1216705 merged by jenkins-bot:

[mediawiki/extensions/TestKitchen@master] Initialize Test Kitchen with revised files from Metrics Platform

https://gerrit.wikimedia.org/r/1216705

Change #1220038 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[mediawiki/extensions/TestKitchen@master] Bring Test Kitchen to parity with Metrics Platform

https://gerrit.wikimedia.org/r/1220038

Change #1220038 merged by jenkins-bot:

[mediawiki/extensions/TestKitchen@master] Bring Test Kitchen to parity with Metrics Platform

https://gerrit.wikimedia.org/r/1220038

Change #1225005 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/mediawiki-config@master] Deploy TestKitchen to testwiki

https://gerrit.wikimedia.org/r/1225005

Change #1216847 merged by jenkins-bot:

[operations/mediawiki-config@master] extension-list: Add Test Kitchen

https://gerrit.wikimedia.org/r/1216847

Mentioned in SAL (#wikimedia-operations) [2026-01-12T10:58:06Z] <ladsgroup@deploy2002> Started scap sync-world: Backport for [[gerrit:1216847|extension-list: Add Test Kitchen (T407806)]]

Mentioned in SAL (#wikimedia-operations) [2026-01-12T11:22:19Z] <ladsgroup@deploy2002> cjming, ladsgroup: Backport for [[gerrit:1216847|extension-list: Add Test Kitchen (T407806)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-01-12T11:36:12Z] <ladsgroup@deploy2002> Finished scap sync-world: Backport for [[gerrit:1216847|extension-list: Add Test Kitchen (T407806)]] (duration: 38m 06s)

Change #1217360 merged by jenkins-bot:

[operations/mediawiki-config@master] Deploy TestKitchen to Beta Cluster

https://gerrit.wikimedia.org/r/1217360

Mentioned in SAL (#wikimedia-operations) [2026-01-12T21:02:32Z] <cjming@deploy2002> Started scap sync-world: Backport for [[gerrit:1217360|Deploy TestKitchen to Beta Cluster (T407806 T407805)]]

Please also alias mw.testKitchen to mw.tk.

Change #1226318 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/puppet@production] Add Test Kitchen maintenance script

https://gerrit.wikimedia.org/r/1226318

Change #1226693 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[mediawiki/extensions/TestKitchen@master] Add alias mw.tk for mw.testKitchen

https://gerrit.wikimedia.org/r/1226693

Change #1225005 merged by jenkins-bot:

[operations/mediawiki-config@master] Deploy TestKitchen to testwiki

https://gerrit.wikimedia.org/r/1225005

Mentioned in SAL (#wikimedia-operations) [2026-01-14T14:39:51Z] <lucaswerkmeister-wmde@deploy2002> Started scap sync-world: Backport for [[gerrit:1225005|Deploy TestKitchen to testwiki (T407806)]]

Mentioned in SAL (#wikimedia-operations) [2026-01-14T14:42:06Z] <lucaswerkmeister-wmde@deploy2002> cjming, lucaswerkmeister-wmde: Backport for [[gerrit:1225005|Deploy TestKitchen to testwiki (T407806)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-01-14T14:49:40Z] <lucaswerkmeister-wmde@deploy2002> Finished scap sync-world: Backport for [[gerrit:1225005|Deploy TestKitchen to testwiki (T407806)]] (duration: 09m 49s)

Change #1226318 merged by RLazarus:

[operations/puppet@production] Add Test Kitchen maintenance script

https://gerrit.wikimedia.org/r/1226318

Change #1227004 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/mediawiki-config@master] Enable Test Kitchen on all prod wikis

https://gerrit.wikimedia.org/r/1227004

TL;DR: after pushing the TK extension to testwiki, things appear to be working as expected 🎉

Pending discussion re: questions appended below, here is a config patch for deploying TK extension to all production wikis - https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1227004

More details on how testwiki was tested

Config for turning on TK extension in testwiki (first production wiki) was deployed early today.

Later this afternoon, the TK maintenance script patch was merged/deployed after which I ran UpdateConfigs on testwiki to fetch the latest experiment configs from the TK API:

mwscript-k8s --follow -- extensions/TestKitchen/maintenance/UpdateConfigs.php --wiki testwiki

https://mpic.wikimedia.org/api/v1/experiments?format=config&authority=mediawiki response:

[
  {
    "name": "we-3-3-4-reading-list-test1",
    "start": "2025-11-12T14:30:00Z",
    "end": "2026-07-01T14:30:00Z",
    "sample_rate": {
      "1": [
        "frwiki",
        "arwiki",
        "idwiki",
        "viwiki",
        "zhwiki"
      ],
      "default": 0
    },
    "groups": [
      "control",
      "treatment"
    ]
  },
  {
    "name": "we-3-3-4-reading-list-test1-en",
    "start": "2025-11-19T14:30:00Z",
    "end": "2026-07-01T14:30:00Z",
    "sample_rate": {
      "1": [
        "enwiki"
      ],
      "default": 0
    },
    "groups": [
      "control",
      "treatment"
    ]
  },
  {
    "name": "growthexperiments-revise-tone",
    "start": "2025-12-16T14:30:00Z",
    "end": "2026-03-31T14:30:00Z",
    "sample_rate": {
      "1": [
        "testwiki"
      ],
      "default": 0
    },
    "groups": [
      "control",
      "treatment"
    ]
  }
]

Tested https://test.wikipedia.org/wiki/Main_Page by logging in and running the following commands in the console:

mw.config.get('wgTestKitchenUserExperiments');

which returned:

and which matches output from $wgMetricsPlatformUserExperiments:

🎊 🎉

Note that the coordinator value is default in the case of TK and xLab in the case of MP

Because I didn't want to pollute data collection for active experiments, I opted next to test further by looking at output on a non-active experiment i.e. foo-bar

mw.testKitchen.getExperiment('foo-bar');

returned:

which is the expected outcome - an UnenrolledExperiment object - note that coordinator = default and the schemaID is web base 2.0.0

I then tested overriding functionality by setting the mpo (i guess we should update this cookie to tko?) cookie:

which returned an expected OverriddenExperiment object - note that coordinator = forced and the subjectId = overridden

And finally I tried to send an event for the foo-bar experiment:

which returned the expected response that the experiment enrollment was overridden so the corresponding event was not sent.

Next Steps/Questions

Presumably we want to test the actual end-to-end (seeing experiment events in EventGate) by sending events from TK using our own experiments (tracked in T414528: Run synthetic experiment using Javascript SDK in Test Kitchen and T414530: Run synthetic experiment using PHP SDK in Test Kitchen).

• Do we want to do this on testwiki alone before we deploy to all prod wikis?
• Are we comfortable enough with the current evidence that TK is working on testwiki to deploy to all prod wikis as is first and then run our experiments?
  • Note that T414644: MediaWikiCronJobFailed arose because the maintenance script to update configs from Test Kitchen expects the TK extension to be everywhere - it should be resolved by enabling TK extension on all prod wikis
    • If we want to test on testwiki first, should the maintenance script be updated temporarily to apply just to testwiki to stop alerts in the meantime? we can revert when TK is enabled on all prod wikis
• It just occurred to me that we should also test data collection for instruments using TK as well - I will spin up those tickets separately. Same questions apply from above to TK-managed instruments
• Is it permissible to send test events on active experiments? i am assuming absolutely not but raising the question out of curiosity - are just a few random events ok if they're coming from testwiki (ack'ing that growthexperiments-revise-tone is 100% on testwiki so not that experiment - maybe the other 2?)? could they be filtered out of data analysis by excluding testwiki events? This would give us confirmation that experiment events are sent as expected via TK SDKs immediately, though at the expense of annoying a product analyst and making their work life more difficult.

cc @JVanderhoop-WMF @phuedx @KReid-WMF @Sfaci
also cc @mpopov for my last question

oof - i keep forgetting about wgTestKitchenExperiments -- i can push config for this tomorrow and see what happens

though for clarification, presumably it's ok to have it in prod settings (assuming it gets deleted after testing) since TK is only on for testwiki right now?

Change #1226693 merged by jenkins-bot:

[mediawiki/extensions/TestKitchen@master] Add alias mw.tk for mw.testKitchen

https://gerrit.wikimedia.org/r/1226693

Change #1227004 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable Test Kitchen on all prod wikis

https://gerrit.wikimedia.org/r/1227004

Mentioned in SAL (#wikimedia-operations) [2026-01-15T14:51:30Z] <phuedx@deploy2002> Started scap sync-world: Backport for [[gerrit:1227004|Enable Test Kitchen on all prod wikis (T407806)]]

Mentioned in SAL (#wikimedia-operations) [2026-01-15T14:53:51Z] <phuedx@deploy2002> cjming, phuedx: Backport for [[gerrit:1227004|Enable Test Kitchen on all prod wikis (T407806)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-01-15T15:05:16Z] <phuedx@deploy2002> Finished scap sync-world: Backport for [[gerrit:1227004|Enable Test Kitchen on all prod wikis (T407806)]] (duration: 13m 46s)

Given that TK extension has been deployed to all prod wikis as of 1/15/26, tested/verified with the JS + PHP SDKs, and the checklist in the description is basically all checked, I'm going to be bold and move this ticket to Done.