Page MenuHomePhabricator

Deploy Recommendation Experiment (Second Experiment) in Chrome Extension
Closed, ResolvedPublic2 Estimated Story Points

Description

Background

We want to develop a second recommendation experiment within the Chrome extension, placing recommendation cards alongside articles using the MoreLike API.
Once the experiment is ready, deploy to the existing Chrome extension.

User Story

As a product team, we need to deploy the new recommendation experiment within the Chrome extension to test its performance and gather user interaction data for further analysis.

Requirements

Deploy the new recommendation experiment to the existing Chrome extension.

Technical Notes:
The extension has separated experiments in folders that mimic ResourceLoader module naming conventions. The first experiment lives in the folder src/content_scripts/vector2022.empty.search . The main.js file in that folder is loaded onto the page by importing it in injector.js and injecting in onto the page via a <script> tag.
In order to disable that experiment, we have to remove it from the injector.js script and replace it with a new script.

injector.js
import main from './vector2022.empty.search/main?script&module'; 👈 // replace this file with a new experiment

const scriptTag = document.createElement( 'script' );
scriptTag.src = chrome.runtime.getURL( main );
scriptTag.setAttribute( 'defer', 'defer' );
scriptTag.setAttribute( 'type', 'module' );
document.body.appendChild( scriptTag );
Acceptance Criteria
  • Successfully deploy the experiment in the Chrome extension.
  • The experiment must be fully functional as per requirements.
  • Instrumentation is set up and tracking correctly.
  • No regression or negative user impact is observed.
  • Flag any issues or unknowns during the deployment process and address them.
Communication

Announcements or internal discussions should be held if necessary to inform stakeholders.

Rollback Plan

Ensure a defined rollback plan is in place if any issues arise in production after deployment.

Event Timeline

KSarabia-WMF renamed this task from Deploy Recommendation Experiment in Chrome Extension to Deploy Recommendation Experiment (Second Experiment) in Chrome Extension.Sep 5 2024, 11:05 PM
Jdrewniak raised the priority of this task from High to Needs Triage.Sep 12 2024, 4:40 PM
Jdrewniak triaged this task as High priority.
Jdrewniak updated the task description. (Show Details)

Is there a list of potential deploy blockers/a checklist? A few things that come to mind (might have already been solved, ignore if so):

  • We should explicitly discuss whether the term of use checkbox is required explicitly: T370209: [EPIC] Experiment: Display article recommendations in more prominent locations
  • Are there plans to run an additional quicksurvey alongside the deploy? Should there be? I would say we can go ahead and deploy without the quicksurvey and take note of the usage rate to make sure we are getting sufficient data. If not, then run quicksurvey
  • Do we plan on informing extension users that they are being switched to a new way of presenting recommendations?
  • Does the extension page need any edits?

At the risk of coaching extension users to seek out a new recommendation feature, I'd recommend we don't explicitly tell them that the experiments are changing outside updating the mini window for the extension, so they can see the changes if they seek them out.

Earlier this week we added a scheduling feature to the extension that lets us enable/disable experiments in a future date.
In hindsight that patch should have been tagged with this ticket:
https://gitlab.wikimedia.org/repos/web/web-experiments-extension/-/commit/b6868fd0ee885e02717379ccf9dbacd0cd262f68
The second experiment was scheduled to be enabled on Sept 24, 19:00 UTC.

However, due to an error in the code I just discovered (forgot to include hours/minutes/seconds in the UTC Date) The experiment will actually start on
Midnight Sept 25th i.e. 2024-09-25T00:00:00Z

// The following date object: 
const todayUTC = new Date( Date.UTC(
	today.getUTCFullYear(),
	today.getUTCMonth(),
	today.getUTCDate()
);

// Should have been
const todayUTC = new Date( Date.UTC(
	today.getUTCFullYear(),
	today.getUTCMonth(),
	today.getUTCDate(),
	today.getUTCHours(),
	today.getUTCMinutes(),
	today.getUTCSeconds()
);