Analytics for SDAW Search Improvements
Open, MediumPublic
Actions

Assigned To

Authored By

	CBogen
	May 4 2022, 12:59 PM

Description

As a product manager for Structured Data, I want to understand how the new search improvements features are being used and how they impact search metrics in general, so that I can make decisions about how to build and/or make changes to the features going forward, and determine their success.

Ideally we can create a dashboard with this information.

Specific analytics questions to answer/acceptance criteria: TBD, but might include things like, how often is quickview opened? How often do users go directly to a section instead of clicking on the main result? How often do users find what they need in quickview and go no further? How do we factor in these changes to our existing search metrics, like search satisfaction?

Metrics to measure:

Quickview metrics (T311166)
Sister projects metrics (T311165)

Related Objects
Search...

Status	Assigned	Task
Resolved	None	T307038 [Placeholder EPIC] SDAW Search Improvements
Open	cchen	T307571 Analytics for SDAW Search Improvements
Resolved	cchen	T311165 Add CTR for sister projects on special:search to search improvements dashboard
Resolved	cchen	T311166 Special:Search Quickview analytics
Resolved	cchen	T307572 [M] Instrumentation for SDAW search improvements
Resolved	cchen	T324707 QA Search preview instrumentation

Event Timeline

CBogen created this task.May 4 2022, 12:59 PM

CBogen added a parent task: T307038: [Placeholder EPIC] SDAW Search Improvements.

CBogen mentioned this in T307572: [M] Instrumentation for SDAW search improvements.May 4 2022, 1:04 PM

We have this analytics ticket btw https://phabricator.wikimedia.org/T306344

CBogen updated the task description. (Show Details)May 4 2022, 1:58 PM

CBogen merged a task: T306344: [Analytics] Metrics for the quickview panel.

Product Analytics will review and prioritize this request during our next board review meeting on 10 May 2022.

CBogen moved this task from Triage to Analytics on the Structured-Data-Backlog board.May 9 2022, 4:26 PM

mpopov assigned this task to cchen.May 10 2022, 5:24 PM

mpopov triaged this task as Medium priority.

mpopov moved this task from Triage to Blocked on the Product-Analytics board.

mpopov added a subtask: T307572: [M] Instrumentation for SDAW search improvements.

cchen updated the task description. (Show Details)May 11 2022, 8:53 AM

Hello does the following also include all the links inside the quick view? If not, it would be good to add that too to see what users are clicking on inside quick view. Are they exploring it further by clicking on links or simply reading what is shown in quick view and moving on to other results?

Interaction Metrics:

CTR on titles/sections/images
Dwell time after quick view panel open

cchen updated the task description. (Show Details)Jun 16 2022, 7:34 AM

@Sneha yes, this means the links/sections/images inside the quick view. we plan to measure interactions for users who click anything in the quick view. and for the users who open quick view panel but don't click, we measure dwell time for user interactions.

In T307571#7995511, @Sneha wrote:

Hello does the following also include all the links inside the quick view? If not, it would be good to add that too to see what users are clicking on inside quick view. Are they exploring it further by clicking on links or simply reading what is shown in quick view and moving on to other results?

Interaction Metrics:

CTR on titles/sections/images

Dwell time after quick view panel open

@cchen at the offsite, we discussed wanting to track whether the visual changes we're making to the way sister projects are displayed (outside of quickview) impacts their CTR. The sister projects are already instrumented using existing data/logging. Should we make a separate ticket to make sure we're paying attention to that, or can we add it to the questions in this ticket?

@CBogen let's make separate tickets then! maybe we can make a separate one for Quickview as well, and use this ticket as a parent ticket for all the changes.

CBogen added a subtask: T311165: Add CTR for sister projects on special:search to search improvements dashboard.Jun 22 2022, 5:55 PM

In T307571#8020752, @cchen wrote:

@CBogen let's make separate tickets then! maybe we can make a separate one for Quickview as well, and use this ticket as a parent ticket for all the changes.

Sounds good! I created T311165 and T311166, made this ticket their parent, and updated this ticket's description to match.

In T307571#8020810, @CBogen wrote:

In T307571#8020752, @cchen wrote:

@CBogen let's make separate tickets then! maybe we can make a separate one for Quickview as well, and use this ticket as a parent ticket for all the changes.

Sounds good! I created T311165 and T311166, made this ticket their parent, and updated this ticket's description to match.