Page MenuHomePhabricator

Deploy config change to start the Paste Check A/B Test
Closed, ResolvedPublic

Description

Deployment timing

Wednesday, 8 October 2025

Tech/News Announcement

This user-notice was handled as part of T399669#11234558

Bucketing criteria

Bucketing Requirements:

  • Bucketing should include both registered and unregistered users at all identified "Participating wikis" below.
  • All users, who are editing a desktop or mobile main namespace page (NS:0), at any of the participating wikis should have a 50% chance of being included/bucketed into the A/B test's control or treatment group.
  • Bucketing should be done on a per-Wikipedia basis. 50% of people within a given wiki should be placed within the control group; 50% should be bucketed in the treatment group
  • The test group should have Paste Check experience enabled while the control group should not
  • People should remain in the same test group for the duration of the test (and across sessions and pages).
NOTE: We need to specify the experiences that will be available in both the test and the control groups. For example, will all wikis in the control group have access to Multi-Check (References)?

Instrumentation-Related Requirements

  • A bucket is applied to these events so we can distinguish all events logged for the control group and the test group within the A/B test.
    • The bucket should be descriptive of the test in case there are overlapping AB tests.
  • The anonymous_user_token field is populated for unregistered users in the test.

Participating wikis

NOTE: List of wikis was finalized via T403680.
IDWiki
1.de.wiki
2.pl.wiki
3.id.wiki
4.vi.wiki
5.tw.wiki
6.simple.wiki
7.it.wiki
8.nl.wiki
9.cs.wiki
10.uk.wiki
11.ru.wiki
12.bn.wiki
13.hi.wiki
14.zh.wiki
15.ar.wiki
16.fa.wiki
17.sw.wiki
18.dag.wiki
19.fon.wiki
20.ca.wiki
21.eu.wiki
22.gl.wiki

Details

Due Date
Oct 8 2025, 7:00 AM
Other Assignee
Ryasmeen
Related Changes in Gerrit:

Event Timeline

ppelberg set Due Date to Oct 8 2025, 7:00 AM.
ppelberg added a project: User-notice.
ppelberg updated the task description. (Show Details)
Restricted Application added a subscriber: alaa. · View Herald TranscriptOct 6 2025, 7:46 PM

Change #1194334 had a related patch set uploaded (by DLynch; author: DLynch):

[operations/mediawiki-config@master] Launch VisualEditor EditCheck paste check a/b test to 22 wikis

https://gerrit.wikimedia.org/r/1194334

Change #1194334 merged by jenkins-bot:

[operations/mediawiki-config@master] Launch VisualEditor EditCheck paste check a/b test to 22 wikis

https://gerrit.wikimedia.org/r/1194334

Mentioned in SAL (#wikimedia-operations) [2025-10-08T20:06:12Z] <tgr@deploy2002> Started scap sync-world: Backport for [[gerrit:1194650|eswiki, commonswiki: lift IP cap for workshop (T406655)]], [[gerrit:1194334|Launch VisualEditor EditCheck paste check a/b test to 22 wikis (T405422)]]

Mentioned in SAL (#wikimedia-operations) [2025-10-08T20:11:18Z] <tgr@deploy2002> tgr, kemayo, anzx: Backport for [[gerrit:1194650|eswiki, commonswiki: lift IP cap for workshop (T406655)]], [[gerrit:1194334|Launch VisualEditor EditCheck paste check a/b test to 22 wikis (T405422)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-10-08T20:19:16Z] <tgr@deploy2002> Finished scap sync-world: Backport for [[gerrit:1194650|eswiki, commonswiki: lift IP cap for workshop (T406655)]], [[gerrit:1194334|Launch VisualEditor EditCheck paste check a/b test to 22 wikis (T405422)]] (duration: 13m 03s)

DLynch added a project: Editing QA.

You can check that these values are being correctly set after VE is launched:

mw.config.get( 'wgVisualEditorConfig' ).editCheckABTest
mw.config.get( 'wgVisualEditorConfig' ).editCheckABTestGroup
Ryasmeen edited projects, added Verified; removed Editing QA.
Ryasmeen subscribed.

Checked the following in de.wiki and pl.wiki:

  1. Bucketing is including both registered and unregistered users.
  2. Checked for users who are editing a desktop or mobile main namespace page (NS:0).
  3. Bucketing is being done on a per-Wikipedia basis. Same user has been observed being placed in different groups for different wikis.
  4. The test group has Paste Check experience enabled while the control group is not.
  5. People does remain in the same test group for the duration of the test (and across sessions and pages).
  6. Verified that anonymous_user_token field is populated for unregistered users in editAttemptStepon desktop.