Measure whether users used the "did you mean" results that we serve to users somehow
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Deskana
	Jul 8 2015, 5:50 PM

Description

We have no idea whether our users are actually using the "Did you mean" results we're serving them. Without quantitative data on this, it's hard to know whether any changes we make will actually serve our users better.

Let's figure out some way of measuring whether users are clicking on the did you mean results we serve them, so that we can perform improvements quickly and measure our success easily.

What I'm looking for to be recorded:

How many results the user was shown for their query
Whether the user was shown a did you mean result
Whether the user clicked on the did you mean result

Statistics that should be computable from this:

Percentage of queries where the user was shown a "Did you mean" result
Percentage of queries where, having been shown a "Did you mean" result, the user clicked on that result

Stakeholder: Search Team, Discovery Department
Benefit: We'll know how well any changes we make to "Did you mean" results actually work, as well as knowing current effectiveness
Estimate: To be added

Details

	Subject	Repo	Branch	Lines +/-
	Measure usage of Special:Search did you mean	mediawiki/extensions/WikimediaEvents	master	+98 -1

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Declined		Ironholds	T111892 Perform analysis of how often the "Did you mean" suggestions are actually used
		Resolved		EBernhardson	T105201 Measure whether users used the "did you mean" results that we serve to users somehow

Event Timeline

• Deskana created this task.Jul 8 2015, 5:50 PM

• Deskana raised the priority of this task from to Medium.

• Deskana updated the task description. (Show Details)

• Deskana added projects: Discovery-ARCHIVED, Discovery-Search (Current work).

• Deskana subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 8 2015, 5:50 PM

• Deskana mentioned this in T105204: Get more data on how many users are getting "Did you mean" suggestions.Jul 8 2015, 5:55 PM

• Deskana renamed this task from Measure effectiveness of the "did you mean" results that we serve to users somehow to Measure whether users used the "did you mean" results that we serve to users somehow.Jul 8 2015, 6:08 PM

• Deskana updated the task description. (Show Details)

• Deskana set Security to None.

• Deskana updated the task description. (Show Details)

• Deskana merged a task: T105204: Get more data on how many users are getting "Did you mean" suggestions.Jul 8 2015, 6:15 PM

Is EL the way to go for this collection?

If a user follows the "did you mean" link to a result, do we consider that a zero-result? Or do we only consider a dead-end to be a zero-result?

In T105201#1439505, @Jdouglas wrote:

Is EL the way to go for this collection?

I think so. But, ultimately, as the product owner I am agnostic to the technical solution as long as it meets the acceptance criteria.

In T105201#1439524, @Jdouglas wrote:

If a user follows the "did you mean" link to a result, do we consider that a zero-result? Or do we only consider a dead-end to be a zero-result?

I would consider the user as having done two searches. The first gave them zero results and the "did you mean". The second one was based on the "did you mean", and may or may not have given them results. Does that answer your question?

Ah, that makes sense, thanks.

• Jdouglas claimed this task.Jul 8 2015, 11:10 PM

• Jdouglas moved this task from Incoming to not in use - please delete on the Discovery-Search (Current work) board.

EBernhardson removed • Jdouglas as the assignee of this task.Jul 30 2015, 4:54 AM

EBernhardson moved this task from not in use - please delete to Incoming on the Discovery-Search (Current work) board.

EBernhardson moved this task from Incoming to not in use - please delete on the Discovery-Search (Current work) board.Aug 17 2015, 7:48 PM

EBernhardson claimed this task.Aug 17 2015, 9:09 PM

Change 232376 had a related patch set uploaded (by EBernhardson):
Measure usage of Special:Search did you mean

https://gerrit.wikimedia.org/r/232376

gerritbot added a project: Patch-For-Review.Aug 18 2015, 8:14 PM

EBernhardson moved this task from not in use - please delete to Needs review on the Discovery-Search (Current work) board.Aug 18 2015, 8:14 PM

TJones mentioned this in T110359: Track clicking on suggestion in "Did you mean" results.Aug 26 2015, 5:44 PM