Page MenuHomePhabricator

Build second recommendation experiment within browser extension
Closed, ResolvedPublic5 Estimated Story Points

Assigned To
Authored By
ovasileva
Aug 29 2024, 10:00 AM
Referenced Files
F57534675: image.png
Sep 24 2024, 6:32 PM
F57534672: image.png
Sep 24 2024, 6:32 PM
F57534412: screenshot 29.png
Sep 24 2024, 4:05 PM
F57534410: screenshot 28.png
Sep 24 2024, 4:05 PM
F57534390: screenshot 27.png
Sep 24 2024, 4:05 PM
F57534383: screenshot 26.png
Sep 24 2024, 4:05 PM
F57534385: screenshot 25.png
Sep 24 2024, 4:05 PM
F57532451: chrome-extension-sidebar-experiment.zip
Sep 23 2024, 6:43 PM

Description

Background

  • We would like to explore recommendations in a few different places on the page, including alongside articles. One of the things we want to explore is reusing the current extension to

User story

  • As a product team, we want the ability to test more recommendation features in a lightweight way, so we can learn quicker

Requirements

  • No changes must be seen by end user while we're building out this functionality. Deployment will be handled separately.
  • At lower screen resolutions it does not show at all.
  • the menu cannot be pinned and is always shown
  • For long titles limit to 2 lines.
  • Build feature as per requirement using MoreLike API. The cards shown should be identically sourced to the empty search experiment.
  • Instrumentation for recommendations alongside articles must be as follows:
  • We want to detect whether clicks come from people using browser extension or not
  • We want to count the number of clicks on recommendations on the side of the article
  • Instrumentation should be distinguishable from the previous experiment - previously we used experiment.emptySearch use experiment.sidebar

Requirement

The recommendation feature should display alongside articles. The feature should not be visible to end users during its development. Once deployed, it must not display at lower screen resolutions, and the menu for recommendations should always be visible and unpinned. Long titles within the recommendations should be limited to two lines. Additionally, the feature must include instrumentation to track clicks on the recommendations, distinguishing whether they come from browser extension users, using the experiment.sidebar event name for tracking.

BDD

Feature: Recommendations Alongside Articles

  Scenario: Recommendations do not display at lower resolutions
    Given the user views a page at a lower screen resolution
    When the page is loaded
    Then recommendations should not be shown

  Scenario: Recommendation titles are limited to two lines
    Given a recommendation has a long title
    When the recommendation is displayed
    Then the title should be limited to two lines

  Scenario: Track clicks on recommendations
    Given the user clicks on a recommendation alongside an article
    When the click is recorded
    Then the click should be tracked using the `experiment.sidebar` event name
    And the instrumentation should detect whether the click came from a browser extension user

Test Steps

Test Case 1: Verify Recommendations Do Not Show at Lower Resolutions

  1. Adjust the screen resolution to a lower setting.
  2. AC1: Confirm that recommendations alongside articles do not display.

Test Case 2: Verify Long Titles are Limited to Two Lines

  1. Visit a page with recommendations that have long titles.
  2. AC2: Confirm that long titles are limited to two lines.

Test Case 3: Verify Click Tracking on Recommendations

  1. Click on a recommendation alongside an article.
  2. AC3: Confirm that the click is tracked using the experiment.sidebar event name.
  3. AC4: Confirm that the instrumentation can detect if the click came from a browser extension user.

Design

image.png (1×2 px, 1 MB)

Acceptance criteria

  • Complete all requirements
  • Flag any unknowns and ask appropriate stakeholders
  • Set up ticket for launching experiment

Communication criteria - does this need an announcement or discussion?

  • Another survey perhaps

Rollback plan

  • What is the rollback plan in production for this task if something goes wrong?

This task was created by Version 1.2.0 of the Web team task template using phabulous

Event Timeline

ovasileva renamed this task from Prepare browser extension for re-use for second experiment to Prepare browser extension for re-use for second recommendation experiment.Aug 29 2024, 10:14 AM
Jdlrobson subscribed.

Jan to expand on questions that came up during our estimation today.

ovasileva renamed this task from Prepare browser extension for re-use for second recommendation experiment to Build second recommendation experiment within browser extension.Sep 3 2024, 5:55 PM
ovasileva updated the task description. (Show Details)
ovasileva updated the task description. (Show Details)
ovasileva added subscribers: JScherer-WMF, Jdrewniak.

@JScherer-WMF to upload new mock and bring to the team for grooming.

My goal for this is the simplest thing I can think of that mimics the content and layout of the empty search recommendations while maintaining the use of those sidebar boxes that we already have in prod. I'll bring this to design review in the morning and probably do one quick iteration before our task sync in the afternoon.

This is where I've landed:

image.png (1×2 px, 1 MB)

image.png (1×2 px, 1 MB)

long text is two lines then truncate

image.png (508×484 px, 71 KB)

Jdlrobson set the point value for this task to 5.Sep 5 2024, 5:38 PM

Marketers: « Oh hey, let's create a browser extension, so we'll have yet another thing to maintain. To display article recommendations. We definitely need a browser extension for this. »

User: « Oh, what a great idea, I'll install a 524th extension on my browser. I like to install bloat. More is better, right? »

Seems so unnecessary.

There are already hyperlinks all over the place. You know, things to go explore other locations.

Not to mention navboxes, portals, "see also" sections, featured articles…

@Od1n: Please post non-constructive or sarcastic comments in other places if you would like to stay active in Phabricator. Thanks for your understanding.

On the contrary, such comments are purposeful. To make people realize when developement is going the wrong way.

Hey @Od1n, thanks for your interest in our work. Let's take a step back. It will be a shame if we continue to discuss the details without making sure that we're on the same page about the basics.

What you see here is a very small part of a larger picture documented in the Annual Plan. You can learn more about the picture on this page: Readers/2024 Reader and Donor Experiences/Content Discovery. I will be really grateful if you look at the "Key Result -> hypothesis" line of thinking, and I'm curious if you have any thoughts about any of these pieces (Key Result or the hypotheses). Let's maybe continue our discussion on this talk page? Thanks!

@Od1n: It is possible to ask for the bigger picture and for decision records instead of posting uninformed sarcastic comments. Please do so in the future. Thanks.

please grab a screenshot and add it here so @JScherer-WMF can look at it.

Disappears when sidebar collapses

Screenshot 2024-09-17 at 1.32.41 PM.png (1×1 px, 987 KB)

On tooltip hover

Screenshot 2024-09-17 at 1.09.49 PM.png (1×2 px, 1007 KB)

On page load

Screenshot 2024-09-17 at 1.09.55 PM.png (1×2 px, 1022 KB)

cc: @JScherer-WMF

Thanks @KSarabia-WMF! It's looking really good. I was thinking we would wrap to a second line then truncate the text with an ellipsis like this:

image.png (512×488 px, 74 KB)

I didn't see any affordance for truncating in the codex component, though, so not sure if this is available to us.

For the tooltip text, let's say

Recommendations based on similar articles

Thanks!

For QA purposes, I'm attaching a dev build of the chrome extension which features the sidebar experiment.
Here are instruction for installing unpacked extension: https://knowledge.workspace.google.com/kb/load-unpacked-extensions-000005962

Test Result - Beta

Status: ❓Need More Info / ❌ FAIL
Environment: Unpacked Extension
OS: macOS
Browser: Chrome
Device: MS MBA
Emulated Device: NA

Test Artifact(s):

Test Steps

Test Case 1: Verify Recommendations Do Not Show at Lower Resolutions

  1. Adjust the screen resolution to a lower setting.
  2. AC1: Confirm that recommendations alongside articles do not display.

1120px and above

screenshot 25.png (1×1 px, 777 KB)

Below 1120px

screenshot 26.png (1×1 px, 707 KB)

Test Case 2: Verify Long Titles are Limited to Two Lines

  1. Visit a page with recommendations that have long titles.
  2. AC2: Confirm that long titles are limited to two lines.

screenshot 27.png (1×1 px, 584 KB)

Test Case 3: Verify Click Tracking on Recommendations

  1. Click on a recommendation alongside an article.
  2. AC3: Confirm that the click is tracked using the experiment.sidebar event name.

the event name is experiment.web.sidebar. I'm assuming this is correct but might as well check.

  1. AC4: Confirm that the instrumentation can detect if the click came from a browser extension user.

This is different than how previous events showed up in the network tab so putting it here to verify this is what was expected.

screenshot 28.png (367×954 px, 63 KB)

screenshot 29.png (406×954 px, 73 KB)

Jdlrobson added a subscriber: Edtadros.

The truncation is not a blocker for deployment, but we do recognize it as a bug. Justin will take a pass and move to sign off to unblock deployment.

I just wanted to see the tooltip, and it looks good. But we still need Spanish translations. Moving back to blocked until we have those.

Ok! @ovasileva checked our Spanish. We're good to go with the translations you have in there already @Jdrewniak

image.png (347×360 px, 44 KB)

image.png (360×286 px, 48 KB)

Now launched! Resolving this ticket.