Background

We would like to deploy our summary experiment in our browser extension

User story

As a browser extension user, I want to see summaries within pages, so that I can get a quick glance of what an article is about

Requirements

Deployment prep:

• The previous experiment must be disabled
• The new experiment should be enabled with a scheduled start and end date
• The version number of the extension should be increased

Experiment smoke test
Before publishing to the Chrome store, we should do a QA run to verify the experiment functionality.

• Popup
  • The new terms-of-service popup appears on install and the terms can be accepted or declined.
  • The summary experiment appears when the terms of service have been accepted.
• Experiment
  • The simplified summaries appear for the following list of pages: https://gitlab.wikimedia.org/repos/web/web-experiments-extension/-/tree/main/src/assets/summaries?ref_type=heads
  • The summaries can be expanded or collapsed
• Instrumentation
  • A request is posted to the statsv endpoint:
    • On every pageview when the extension is enabled: enabled
    • On every pageview where a summary is available: available
    • When a user opens a summary: open
    • When a user clicks the 'yes' button: yes
    • When a user clicks the 'no' button: no

BDD

Feature: Summary Experiment Deployment

Scenario: Verify Terms of Service Popup
Given the extension is installed
When the user launches the extension
Then the terms-of-service popup should appear
And the user can accept or decline the terms.

Scenario: Verify Summary Display
Given the terms of service have been accepted
And the user is viewing one of the specified pages
When the extension is active
Then simplified summaries should appear
And the summaries should be expandable or collapsible.

Scenario: Verify Instrumentation Events
Given the extension is active
When the user interacts with the summary experiment
Then the following events should be logged to the statsv endpoint:
| Action        | Event     |
| Pageview      | enabled   |
| Summary Found | available |
| Summary Open  | open      |
| Yes Clicked   | yes       |
| No Clicked    | no        |

Test Steps

Test Case 1: Verify Terms of Service Popup

1. Install the browser extension.
2. Launch the extension.
3. AC1: Confirm that the terms-of-service popup appears and users can accept or decline the terms.

Test Case 2: Verify Summary Display

1. Accept the terms of service in the popup.
2. Navigate to a page listed in the repository: https://gitlab.wikimedia.org/repos/web/web-experiments-extension/-/tree/main/src/assets/summaries?ref_type=heads.
3. AC2: Confirm that simplified summaries appear on the page.
4. Expand and collapse the summaries.
5. AC3: Verify that the summaries expand and collapse correctly.

Test Case 3: Verify Instrumentation Events

1. Enable the extension and browse a few pages.
2. Open the Chrome DevTools and monitor network requests to the statsv endpoint.
3. Perform the following actions:
   • View pages with and without summaries.
   • Open a summary.
   • Click the 'yes' and 'no' buttons.
4. AC4: Confirm that the following events are logged correctly:
   • enabled on every pageview when the extension is active.
   • available on every pageview where a summary is available.
   • open when a summary is opened.
   • yes when the 'yes` button is clicked.
   • no when the 'no` button is clicked.

Acceptance criteria

• All subtasks of the epic must be resolved before deployment
• Get a green light from @ovasileva

Communication criteria - does this need an announcement or discussion?

• Add communication criteria

Rollback plan

• The extension build artifact can be rolled back to a previous version through the Chrome store UI.

QA Results - Beta

This task was created by Version 1.2.0 of the Web team task template using phabulous