Disable search A/B test
Closed, ResolvedPublic1 Estimated Story Points
Actions

Assigned To

Authored By

	Jdlrobson-WMF
	Mar 19 2025, 5:39 PM

Description

Background

We should disable the search A/B test on mobile before we go to the offsite.

User story

Add user story in the format: “As a [persona], I want to [X], so that [Y]”

Requirements

Confirm disable date with Jennifer 03/24
Disable A/B test code
Make sure no streams are being logged to

BDD

For QA engineer to fill out

Test Steps

For QA engineer to fill out

Design

Add mockups and design requirements

Acceptance criteria

Add acceptance criteria

Communication criteria - does this need an announcement or discussion?

Add communication criteria

Rollback plan

What is the rollback plan in production for this task if something goes wrong?

Sign off steps

Make sure a task exists for removing the A/B test schema altogether and comment in ticket pointing to it when resolving.

This task was created by Version 1.2.0 of the Web team task template using phabulous

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	Disable Search AB Test	operations/mediawiki-config	master	+4 -17

Customize query in gerrit

Related Objects

Mentioned Here: T388719: Clean up Search AB test code

Event Timeline

Jdlrobson-WMF created this task.Mar 19 2025, 5:39 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 19 2025, 5:39 PM

Jdlrobson-WMF triaged this task as Medium priority.Mar 19 2025, 5:40 PM

Jdlrobson-WMF updated the task description. (Show Details)

Jdlrobson-WMF set the point value for this task to 1.

Jdlrobson-WMF moved this task from Incoming to Q3 Sprint 5 (Mar 19, 2025 - Apr 2, 2025) on the Web-Team board.

Jdlrobson-WMF edited projects, added Web-Team (Q3 Sprint 5 (Mar 19, 2025 - Apr 2, 2025)); removed Web-Team.

bwang claimed this task.Mar 21 2025, 8:45 PM

bwang moved this task from Ready for development to In progress on the Web-Team (Q3 Sprint 5 (Mar 19, 2025 - Apr 2, 2025)) board.

bwang updated the task description. (Show Details)Mar 21 2025, 9:33 PM

Change #1130652 had a related patch set uploaded (by Bernard Wang; author: Bernard Wang):

[operations/mediawiki-config@master] Disable Search AB Test

https://gerrit.wikimedia.org/r/1130652

gerritbot added a project: Patch-For-Review.Mar 24 2025, 4:33 PM

Change #1130652 merged by jenkins-bot:

[operations/mediawiki-config@master] Disable Search AB Test

https://gerrit.wikimedia.org/r/1130652

Mentioned in SAL (#wikimedia-operations) [2025-03-24T20:30:28Z] <brennen@deploy1003> Started scap sync-world: Backport for [[gerrit:1130652|Disable Search AB Test (T389399)]]

Maintenance_bot removed a project: Patch-For-Review.Mar 24 2025, 8:31 PM

Mentioned in SAL (#wikimedia-operations) [2025-03-24T20:35:16Z] <brennen@deploy1003> bwang, brennen: Backport for [[gerrit:1130652|Disable Search AB Test (T389399)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-03-24T20:54:41Z] <brennen@deploy1003> Finished scap sync-world: Backport for [[gerrit:1130652|Disable Search AB Test (T389399)]] (duration: 24m 12s)

Jdlrobson-WMF updated the task description. (Show Details)Mar 25 2025, 6:47 PM

Make sure no streams are being logged to

Events are still coming in.. albeit at a lower frequency according to superset:
SELECT count(*) from product_metrics_web_base_search_ab_test_clicks WHERE day = 25 and month = 3 and hour = 10

7740

SELECT count(*) from product_metrics_web_base_search_ab_test_clicks WHERE day = 24 and month = 3 and hour = 10

385283

I assume pages in cache, I haven't got time to verify this right now, but this is the last thing to do before moving to sign off.

theres still some events coming in for 3/27. seems to be tapering off, is that enough to sign off? @Jdlrobson

bwang removed bwang as the assignee of this task.Mar 27 2025, 3:57 PM

bwang moved this task from In progress to Sign-off on the Web-Team (Q3 Sprint 5 (Mar 19, 2025 - Apr 2, 2025)) board.

bwang subscribed.

Yeh sounds good!

Make sure a task exists for removing the A/B test schema altogether and comment in ticket pointing to it when resolving.

Does this exist? If so I can resolve.

Jdlrobson-WMF claimed this task.Mar 27 2025, 5:23 PM

@Jdlrobson-WMF yep! its in the sprint backlog https://phabricator.wikimedia.org/T388719

Jdlrobson-WMF closed this task as Resolved.Mar 27 2025, 6:01 PM