{GOAL] Run the non-ui survey with new recommendations
Closed, ResolvedPublic0 Estimated Story Points
Actions

Assigned To

Authored By

	JScherer-WMF
	Sep 12 2024, 10:16 PM

Description

Background

Using the output of T379240: Generate API list for recommended content we will run new survey.

The goal is to see how the performance of a recommendation engine based on an article's position in the knowledge network and hand-curated recommendations from community contributors compares to the article content based APIs we tested in the first experiment.

User story

As a member of the web team, I want to know if any other sources of recommendations outperform the 3 APIs in the non-UI experiment.

Requirements

APIs to use: output of T379240
We would like:

3 responses from each API (Minimum of 4 responses, maximum of 12)
6 questions? (randomize if we can?) (maybe 5?)

BDD

For QA engineer to fill out

Test Steps

For QA engineer to fill out

Design

n/a

Acceptance criteria

Add acceptance criteria

Communication criteria - does this need an announcement or discussion?

Add communication criteria

Rollback plan

What is the rollback plan in production for this task if something goes wrong?

This task was created by Version 1.2.0 of the Web team task template using phabulous

Related Objects
Search...

Status	Subtype	Assigned	Task
Declined	Design	None	T370209 [EPIC] Experiment: Display article recommendations in more prominent locations
Declined	Spike	None	T370523 [EPIC] Run UI-less experiment comparing API recommendation quality
Resolved		ovasileva	T374669 {GOAL] Run the non-ui survey with new recommendations
Resolved		ovasileva	T379240 Generate API list for recommended content
Resolved		ovasileva	T379241 Set up quicksurveys for non-UI experiment pt 2

Event Timeline

JScherer-WMF created this task.Sep 12 2024, 10:16 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 12 2024, 10:16 PM

Jdlrobson triaged this task as High priority.Sep 13 2024, 8:56 PM

Jdlrobson lowered the priority of this task from High to Medium.

Jdlrobson moved this task from Needs analysis to Needs design on the Web-Team-Backlog-Archived (FY2024-25 Q1 Sprint 6) board.

ovasileva added a parent task: T370523: [EPIC] Run UI-less experiment comparing API recommendation quality.Sep 17 2024, 10:43 AM

blocked on results of the first survey run:

https://phabricator.wikimedia.org/T374670

ovasileva edited projects, added Web-Team-Backlog-Archived (FY2024-25 Q2 Sprint 1); removed Web-Team-Backlog-Archived (FY2024-25 Q1 Sprint 6).Sep 23 2024, 5:31 PM

Moving to sprint 2 and blocked.

Jdlrobson moved this task from Needs analysis to Blocked on the Web-Team-Backlog-Archived (FY2024-25 Q2 Sprint 2) board.Oct 10 2024, 4:51 PM

ovasileva moved this task from Backlog to Recommendations - APIs on the FY2024-25 KR 3.1 Content Discovery board.Oct 10 2024, 9:07 PM

Jdlrobson reassigned this task from JScherer-WMF to ovasileva.Oct 17 2024, 4:50 PM

MGerlach mentioned this in T369292: Research support for hypothesis in KR WE.3.1 (Q2).Oct 18 2024, 3:48 PM

meeting notes: @ovasileva unclear on next steps and status of rerunning the survey.

ovasileva raised the priority of this task from Medium to High.Oct 25 2024, 4:17 PM

ovasileva mentioned this in T374670: Analyze the results of the non-ui surveys.

ovasileva edited projects, added Web-Team-Backlog-Archived (FY2024-25 Q2 Sprint 3); removed Web-Team-Backlog-Archived (FY2024-25 Q2 Sprint 2).

ovasileva moved this task from Needs analysis to Ready for Development on the Web-Team-Backlog-Archived (FY2024-25 Q2 Sprint 3) board.Oct 28 2024, 5:01 PM

ovasileva lowered the priority of this task from High to Medium.Oct 29 2024, 4:20 PM

ovasileva raised the priority of this task from Medium to High.Oct 29 2024, 6:41 PM

ovasileva updated the task description. (Show Details)

Will be run next week.

Moving to sprint 4 for launching experiment. First we need to get the API results, tracked in T379240: Generate API list for recommended content, will also mark as subtask

ovasileva added a parent task: T379240: Generate API list for recommended content.Nov 7 2024, 12:31 PM

ovasileva removed a parent task: T379240: Generate API list for recommended content.

ovasileva added a subtask: T379240: Generate API list for recommended content.

ovasileva edited projects, added Web-Team-Backlog-Archived (FY2024-25 Q2 Sprint 4); removed Web-Team-Backlog-Archived (FY2024-25 Q2 Sprint 3).Nov 7 2024, 3:14 PM

Jdrewniak reassigned this task from ovasileva to Jdlrobson.Nov 7 2024, 6:47 PM

Jdrewniak added a subscriber: ovasileva.

Jdlrobson renamed this task from Run the non-ui survey with two different recommendation engines to Run the non-ui survey with new recommendations.Nov 7 2024, 6:48 PM

Jdlrobson reassigned this task from Jdlrobson to ovasileva.

Jdlrobson updated the task description. (Show Details)

Jdlrobson moved this task from Needs analysis to Ready for estimation on the Web-Team-Backlog-Archived (FY2024-25 Q2 Sprint 4) board.Nov 12 2024, 5:17 PM

NBaca-WMF added a project: Web-Team.Nov 13 2024, 2:47 PM

NBaca-WMF set the point value for this task to 0.

NBaca-WMF moved this task from Incoming to FY2024-25 Q2 Sprint 4 (Nov 13 - Nov 26, 2024) on the Web-Team board.Nov 13 2024, 6:28 PM

NBaca-WMF edited projects, added Web-Team (FY2024-25 Q2 Sprint 4 (Nov 13 - Nov 26, 2024)); removed Web-Team.

Jdlrobson changed the status of subtask T379241: Set up quicksurveys for non-UI experiment pt 2 from Open to Stalled.Nov 20 2024, 5:19 PM

Dupe of T379241

ovasileva reopened this task as Open.Nov 26 2024, 5:58 PM

ovasileva added a project: Web-Team-Roadmap.

Jdlrobson renamed this task from Run the non-ui survey with new recommendations to {GOAL] Run the non-ui survey with new recommendations.Nov 26 2024, 6:00 PM

Jdlrobson removed projects: Web-Team (FY2024-25 Q2 Sprint 4 (Nov 13 - Nov 26, 2024)), Web-Team-Backlog-Archived (FY2024-25 Q2 Sprint 4).Nov 26 2024, 6:39 PM

ovasileva closed subtask T379240: Generate API list for recommended content as Resolved.Dec 2 2024, 10:20 AM

SToyofuku-WMF changed the status of subtask T379241: Set up quicksurveys for non-UI experiment pt 2 from Stalled to Open.Dec 2 2024, 7:00 PM

ovasileva closed subtask T379241: Set up quicksurveys for non-UI experiment pt 2 as Resolved.Dec 12 2024, 1:41 PM

Jdrewniak closed this task as Resolved.Aug 15 2025, 4:49 PM

{GOAL] Run the non-ui survey with new recommendationsClosed, ResolvedPublic0 Estimated Story PointsActions