Background

In T371294: [Spike 2 days] Identify delivery mechanism pt 2 we discussed delivery mechanisms for a UI experiment on search and narrowed these down to two options (with a potential third option). This ticket will contain the selection of the final delivery mechanism and the development of the experiment itself

User story

• As a product team, we want to know whether adding recommendations into search will benefit users so that we know whether to invest our time into this idea
• As a reader, I want the ability to test cool new features for Wikipedia, so that the site will work based on my needs and feedback

Requirement

Develop and implement an experiment to track search recommendations displayed to users via the Wikipedia Article Recommender Chrome extension. This experiment should present recommendations in the search box of English and Spanish Wikipedia before a user types a query, and all standard search functionality must remain unaffected.

The experiment designed should allow us to track the metrics defined and instrumented in T372780: Instrument empty state of search experiment
The experiment should allow us to collect the data above also for the related pages feature.

BDD

Feature: Search Recommendation Experiment

  Scenario: Show search recommendations before typing
    Given the user installs the Chrome extension on English or Spanish Wikipedia
    When the user clicks into the search box
    Then three search recommendations should appear if the query is blank

  Scenario: Hide search recommendations upon interaction
    Given the search box is active with recommendations displayed
    When the user starts typing a query or clicks outside the box
    Then the recommendations should disappear

  Scenario: Standard search behavior unaffected
    Given the user uses standard search functionality
    When typing a query or selecting a suggestion
    Then the search function should work as expected without interruption

Test Steps

Since this is an experimental feature, the scope of QA test should be to verify this functionality in a way that matches the expected userflow for this experiment.
This experiment has taken the form of a chrome extension: The Wikipedia Article Recommender. This experiment will be advertised to English and Spanish Wikipedia.

This extension presents users with search recommendations before they start typing in the search box in the header of Vector 2022.

Please verify the following:

• The Chome extension can be installed via this link https://chromewebstore.google.com/detail/wikipedia-article-recomme/hocmncobpajkkegkmpmllabogiiplbke?authuser=0&hl=en
• After installing the chrome extension, on English or Spanish Wikipedia (production), the following should happen:
  • When activating (clicking into) the search box in the header, the user is presented with 3 search recommendations before they start typing a search query.
  • The search recommendations only appear when the query is blank, and the search box is active.
  • Clicking off the search box or entering the query should hide the search recommendations
• Standard search activity, i.e. typing a query, clicking the search button, or clicking a search suggestion, should not be affected by this extension.

Test Case 1: Verify Installation and Search Recommendations

1. Install the Chrome extension from this link.
2. Navigate to English or Spanish Wikipedia (production).
3. Click into the search box in the header.
4. AC1: Confirm that three search recommendations appear if the search box is blank.

Test Case 2: Verify Search Recommendations Disappear on Interaction

1. Click off the search box or start typing a query.
2. AC2: Confirm that the recommendations disappear.

Test Case 3: Verify Standard Search Functionality

1. Type a query in the search box and press Enter or click a search suggestion.
2. AC3: Confirm that the standard search behavior functions without any issues from the extension.

Design

Acceptance criteria

• Select one of the following delivery mechanisms after discussing with the team
  • • Browser Extension**
  • Gadget

We selected to proceed with the browser extension during a review of options.

• Build out browser extension according to design above
• Recommendations list should use MoreLike API

Communication criteria - does this need an announcement or discussion?

• Find a way to enroll participants

Rollback plan

• n/a

This task was created by Version 1.2.0 of the Web team task template using phabulous

QA Results - Prod