Page MenuHomePhabricator

Enable sticky header A/B test for idwiki + viwiki
Closed, ResolvedPublic2 Estimated Story Points

Description

NOTE: Scheduled for 15th August 2022

We will enable the A/B test that shows the edit icon in the sticky header for idwiki + viwiki treatment groups.

Acceptance criteria

  • T312573 has been deployed to the target wikis
  • idwiki, viwiki: Control is no sticky header, test1 is sticky header without edit button, test2 is sticky header with edit button

Sign off steps

  • Make sure there is a task to disable the A/B test for idwiki + viwiki << T315264

Developer notes

'wgVectorWebABTestEnrollment' => [
	...
	'idwiki' => [
		'name' => 'vector.sticky_header_edit',
		'enabled' => true,
		'buckets' => [
			'unsampled' => [
				'samplingRate' => 0
			],
			'control' => [
				'samplingRate' => 0.5
			],
			'stickyHeaderEditButtonTreatment' => [
				'samplingRate' => 0.5
			],
		],
	],
	'viwiki' => [
		'name' => 'vector.sticky_header_edit',
		'enabled' => true,
		'buckets' => [
			'unsampled' => [
				'samplingRate' => 0
			],
			'control' => [
				'samplingRate' => 0.5
			],
			'stickyHeaderEditButtonTreatment' => [
				'samplingRate' => 0.5
			],
		],
	],
],
$wgVectorStickyHeader = [
	"logged_in" =>  true,
	"logged_out" => false,
];
$wgVectorStickyHeaderEdit = [
	"logged_in" => true,
	"logged_out" => false,
];

QA steps

See QA steps in T312573 -- instead of testing on a beta cluster page, production pages should be tested.

Sample pages for idwiki and viwiki:

Event Timeline

ovasileva triaged this task as Medium priority.Jul 11 2022, 8:46 AM

Change 821310 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/mediawiki-config@master] Enable sticky header edit A/B test for idwiki + viwiki

https://gerrit.wikimedia.org/r/821310

Change 821310 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable sticky header edit A/B test for idwiki + viwiki

https://gerrit.wikimedia.org/r/821310

Mentioned in SAL (#wikimedia-operations) [2022-08-15T20:12:08Z] <cjming@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:821310|Enable sticky header edit A/B test for idwiki + viwiki (T312295)]] (duration: 03m 30s)

Change 823268 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/mediawiki-config@master] Update sticky header config for idwiki, viwiki A/B experiment

https://gerrit.wikimedia.org/r/823268

Change 823268 merged by jenkins-bot:

[operations/mediawiki-config@master] Update sticky header config for idwiki, viwiki A/B experiment

https://gerrit.wikimedia.org/r/823268

hi @jwang - just a note for you when you do your analysis for this experiment -- after some hiccups with getting the config correct after enabling the test, I was able to verify the different buckets on prod starting 8/16/22 21:21 (UTC). So any data before then can and probably should be excluded. Let me know if you have any Qs.

hi @jwang - just a note for you when you do your analysis for this experiment -- after some hiccups with getting the config correct after enabling the test, I was able to verify the different buckets on prod starting 8/16/22 21:21 (UTC). So any data before then can and probably should be excluded. Let me know if you have any Qs.

cc @MNeisler

ovasileva claimed this task.