Page MenuHomePhabricator

Clean up Search AB test code
Closed, ResolvedPublic3 Estimated Story Points

Description

Background

Now that the AB test is over, we want to make sure we remove the code we wrote. This includes:

  • Hooks firing in MobileFrontend and RelatedArticles
  • Bucketing code in RelatedArticles
  • Instrumentation in WikimediaEvents
  • Eventstream configuration
  • AB flag configuration
  • AB test schema
  • all other styles/js/html we wrote for this code

User story

  • As a developer, we want to avoid accumulating tech debt and unused code so we can deliver features faster

Requirements

  • No visual changes
  • All code and references to the search Ab test code are removed

BDD

Feature: Mobile search continues to function after removing AB test code

  Scenario: Search field interaction functions as before
    Given I am an anonymous user on mobile
    When I tap the search field
    Then the standard type-ahead search suggestions should appear
    And no empty-state recommendations are shown

  Scenario: Search results display correctly after typing
    Given I enter a search query in the search field
    When the results populate
    Then I see relevant results with no visual regressions or JS errors

Test Steps

Test Case 1: Verify search behaves normally without A/B test features

  1. Visit https://en.m.wikipedia.beta.wmflabs.org on a mobile viewport.
  2. Tap the search bar on the main page.
  3. Type in a search term like “Dog”.
  4. Confirm that standard type-ahead suggestions appear.
  5. AC1: Confirm no empty search recommendations are shown.
  6. AC2: Confirm type-ahead results show as expected.
  7. AC3: Confirm no UI regressions or console errors occur.

Design

Acceptance criteria

  • No search ab test events are firing
  • no search ab enrollment is happening

QA Results - Prod

ACStatusDetails
1T388719#10812626
2T388719#10812626
3T388719#10812626

Event Timeline

Jdlrobson-WMF changed the task status from Open to In Progress.Mar 12 2025, 10:09 PM
Jdlrobson-WMF assigned this task to bwang.
Jdlrobson-WMF triaged this task as Medium priority.
Jdlrobson-WMF moved this task from Incoming to Q3 on the Web-Team board.
Jdlrobson-WMF subscribed.

Could you please start an estimation thread?

Jdlrobson set the point value for this task to 3.
Jdlrobson moved this task from Q3 to Sprint Backlog on the Web-Team board.

Moving this one to Q4 since we're still collecting data going into sprint 5

Jdlrobson-WMF changed the task status from In Progress to Open.Mar 31 2025, 5:16 PM
ovasileva raised the priority of this task from Medium to High.Apr 17 2025, 1:22 PM

Change #1138859 had a related patch set uploaded (by Bernard Wang; author: Bernard Wang):

[operations/mediawiki-config@master] Remove Search AB test config

https://gerrit.wikimedia.org/r/1138859

Change #1138860 had a related patch set uploaded (by Bernard Wang; author: Bernard Wang):

[mediawiki/extensions/MobileFrontend@master] Remove Search AB test code

https://gerrit.wikimedia.org/r/1138860

Change #1138860 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontend@master] Remove Search AB test code

https://gerrit.wikimedia.org/r/1138860

Change #1139126 had a related patch set uploaded (by Bernard Wang; author: Bernard Wang):

[mediawiki/extensions/RelatedArticles@master] Remove Search AB test code

https://gerrit.wikimedia.org/r/1139126

bwang removed bwang as the assignee of this task.Apr 25 2025, 8:35 PM

Change #1139126 merged by jenkins-bot:

[mediawiki/extensions/RelatedArticles@master] Remove Search AB test code

https://gerrit.wikimedia.org/r/1139126

Change #1138859 merged by jenkins-bot:

[operations/mediawiki-config@master] Remove Search AB test config

https://gerrit.wikimedia.org/r/1138859

Mentioned in SAL (#wikimedia-operations) [2025-04-28T20:20:22Z] <bd808@deploy1003> Started scap sync-world: Backport for [[gerrit:1137087|dblists: Add sul.dbexpr and generated sul.dblist (T392142)]], [[gerrit:1139474|Design Research Participant Survey: Increase Coverage (T392325)]], [[gerrit:1138859|Remove Search AB test config (T388719)]]

Mentioned in SAL (#wikimedia-operations) [2025-04-28T20:25:02Z] <bd808@deploy1003> dani, bwang, bd808: Backport for [[gerrit:1137087|dblists: Add sul.dbexpr and generated sul.dblist (T392142)]], [[gerrit:1139474|Design Research Participant Survey: Increase Coverage (T392325)]], [[gerrit:1138859|Remove Search AB test config (T388719)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-04-28T20:34:57Z] <bd808@deploy1003> Finished scap sync-world: Backport for [[gerrit:1137087|dblists: Add sul.dbexpr and generated sul.dblist (T392142)]], [[gerrit:1139474|Design Research Participant Survey: Increase Coverage (T392325)]], [[gerrit:1138859|Remove Search AB test config (T388719)]] (duration: 14m 34s)

Jdlrobson-WMF updated the task description. (Show Details)

I think this is done? But not 100% sure.

Jdlrobson-WMF updated the task description. (Show Details)

Bernard to update A/C to be clearer and then pass to me.

Change #1141954 had a related patch set uploaded (by Bernard Wang; author: Bernard Wang):

[mediawiki/extensions/WikimediaEvents@master] Remove Search recommendations AB test code

https://gerrit.wikimedia.org/r/1141954

Change #1141989 had a related patch set uploaded (by Jdlrobson; author: Jdlrobson):

[mediawiki/extensions/RelatedArticles@master] Remove A/B test configuration

https://gerrit.wikimedia.org/r/1141989

Jdlrobson-WMF updated the task description. (Show Details)
Jdlrobson-WMF updated the task description. (Show Details)

Change #1141954 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] Remove Search recommendations AB test code

https://gerrit.wikimedia.org/r/1141954

Change #1141989 merged by jenkins-bot:

[mediawiki/extensions/RelatedArticles@master] Remove A/B test configuration and associated styles

https://gerrit.wikimedia.org/r/1141989

Edtadros subscribed.

Test Steps

Status: ✅ PASS
Environment: enwiki
OS: macOS
Browser: Chrome Canary
Device: MS

Test Case 1: Verify search behaves normally without A/B test features

  1. Visit https://en.m.wikipedia.org on a mobile viewport.
  2. Tap the search bar on the main page.
  3. Type in a search term like “Dog”.
  4. Confirm that standard type-ahead suggestions appear.
  5. ✅AC1: Confirm no empty search recommendations are shown.
  6. ✅AC2: Confirm type-ahead results show as expected.
  7. ✅AC3: Confirm no UI regressions or console errors occur.

Disregard the spacing of the search button. I verified this on a physical iPhone for both Safari and the Chrome browser. This looks like an issue with the mobile "emulation" on the Chrome Canary browser.

screenshot 74.mov.gif (1×978 px, 960 KB)