Background
See parent ticket
User story
- As a product team, we want to know whether adding recommendations into search will benefit users so that we know whether to invest our time into this idea
Requirements
The experiment designed should allow us to track the following:
Quantitative metrics - high priority
- Impressions - can be search sessions initiated (clicks within the search bar) OR some way that we've detected that we're showing a suggestion
- Clicks on suggestions
- Whether a given pageview is using the extension
The experiment should allow us to survey participants on the following:
The experiment should allow us to collect the data above also for the related pages feature.
Requirement
The search experiment should track the following metrics: impressions (when suggestions are shown), clicks on suggestions, and additional clicks. The system should ensure that only the first click fires the "click" event, and subsequent clicks fire the "clickAdditional" event. The experiment should allow verification via the Chrome DevTools Console.
BDD
Feature: Instrument Search Experiment with Event Tracking
Scenario: Track impression events
Given the user opens the search box
When suggestions are shown
Then an impression event should fire each time suggestions appear
And the impression event should be logged as counter.MediaWiki.experiment.web.emptySearch.{dbname}.impression
Scenario: Track click events on suggestions
Given the user clicks on a suggestion in the search box
When the first click occurs
Then a click event should fire
And the click event should be logged as counter.MediaWiki.experiment.web.emptySearch.{dbname}.click
Scenario: Track additional clicks on suggestions
Given the user clicks a suggestion multiple times
When subsequent clicks occur
Then a clickAdditional event should fire
And the clickAdditional event should be logged as counter.MediaWiki.experiment.web.emptySearch.{dbname}.clickAdditional
Scenario: First click triggers click event, additional clicks trigger clickAdditional
Given the user clicks a suggestion in the search box
When the first click occurs
Then only the first click should fire a click event
And subsequent clicks should fire clickAdditional eventsTest Steps
Test Case 1: Verify Impression Events
- Open the search box and view the suggestions.
- Close the search box and reopen it.
- Open the Chrome DevTools Console and verify the event: counter.MediaWiki.experiment.web.emptySearch.{dbname}.impression.
- AC1: Confirm that an impression event fires each time suggestions appear.
Test Case 2: Verify First Click Event
- Click a suggestion in the search box.
- Open the Chrome DevTools Console and verify the event: counter.MediaWiki.experiment.web.emptySearch.{dbname}.click.
- AC2: Confirm that a click event fires on the first click.
Test Case 3: Verify Additional Click Events
- Click the same suggestion multiple times.
- Open the Chrome DevTools Console and verify the event: counter.MediaWiki.experiment.web.emptySearch.{dbname}.clickAdditional.
- AC3: Confirm that the first click fires the click event and all subsequent clicks fire the clickAdditional event.
Test Case 4: First Click Fires Once
- Open the search box and click a suggestion.
- Confirm that any additional clicks on the same suggestion fire the clickAdditional event.
- AC4: Confirm that only the first click fires the click event.
Design
n/a
Acceptance criteria
Communication criteria - does this need an announcement or discussion?
n/a
Rollback plan
- n/a
This task was created by Version 1.2.0 of the Web team task template using phabulous
QA Results - Prod
| AC | Status | Details |
|---|---|---|
| 1 | ✅ | T372780#10128062 |
| 2 | ✅ | T372780#10128062 |
| 3 | ✅ | T372780#10128062 |
| 4 | ✅ | T372780#10128062 |
