Background
See parent ticket
User story
- As a product team, we want to know whether adding recommendations into search will benefit users so that we know whether to invest our time into this idea
Requirements
The experiment designed should allow us to track the following:
Quantitative metrics - high priority
- Impressions - can be search sessions initiated (clicks within the search bar) OR some way that we've detected that we're showing a suggestion
- Clicks on suggestions
- Whether a given pageview is using the extension
The experiment should allow us to survey participants on the following:
The experiment should allow us to collect the data above also for the related pages feature.
Requirement
The search experiment should track the following metrics: impressions (when suggestions are shown), clicks on suggestions, and additional clicks. The system should ensure that only the first click fires the "click" event, and subsequent clicks fire the "clickAdditional" event. The experiment should allow verification via the Chrome DevTools Console.
BDD
Feature: Instrument Search Experiment with Event Tracking Scenario: Track impression events Given the user opens the search box When suggestions are shown Then an impression event should fire each time suggestions appear And the impression event should be logged as counter.MediaWiki.experiment.web.emptySearch.{dbname}.impression Scenario: Track click events on suggestions Given the user clicks on a suggestion in the search box When the first click occurs Then a click event should fire And the click event should be logged as counter.MediaWiki.experiment.web.emptySearch.{dbname}.click Scenario: Track additional clicks on suggestions Given the user clicks a suggestion multiple times When subsequent clicks occur Then a clickAdditional event should fire And the clickAdditional event should be logged as counter.MediaWiki.experiment.web.emptySearch.{dbname}.clickAdditional Scenario: First click triggers click event, additional clicks trigger clickAdditional Given the user clicks a suggestion in the search box When the first click occurs Then only the first click should fire a click event And subsequent clicks should fire clickAdditional events
Test Steps
Test Case 1: Verify Impression Events
- Open the search box and view the suggestions.
- Close the search box and reopen it.
- Open the Chrome DevTools Console and verify the event: counter.MediaWiki.experiment.web.emptySearch.{dbname}.impression.
- AC1: Confirm that an impression event fires each time suggestions appear.
Test Case 2: Verify First Click Event
- Click a suggestion in the search box.
- Open the Chrome DevTools Console and verify the event: counter.MediaWiki.experiment.web.emptySearch.{dbname}.click.
- AC2: Confirm that a click event fires on the first click.
Test Case 3: Verify Additional Click Events
- Click the same suggestion multiple times.
- Open the Chrome DevTools Console and verify the event: counter.MediaWiki.experiment.web.emptySearch.{dbname}.clickAdditional.
- AC3: Confirm that the first click fires the click event and all subsequent clicks fire the clickAdditional event.
Test Case 4: First Click Fires Once
- Open the search box and click a suggestion.
- Confirm that any additional clicks on the same suggestion fire the clickAdditional event.
- AC4: Confirm that only the first click fires the click event.
Design
n/a
Acceptance criteria
Communication criteria - does this need an announcement or discussion?
n/a
Rollback plan
- n/a
This task was created by Version 1.2.0 of the Web team task template using phabulous
QA Results - Prod
AC | Status | Details |
---|---|---|
1 | ✅ | T372780#10128062 |
2 | ✅ | T372780#10128062 |
3 | ✅ | T372780#10128062 |
4 | ✅ | T372780#10128062 |