Analyze wbsearchentities AB test from nov/doc
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	EBernhardson
	Dec 3 2018, 5:17 PM

Description

From the work in T209402 we started an AB test last thursday. It should be turned off on dec 6th and the results reviewed. Metrics:

% of users satisfied after n characters
click position
session abandonment

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	Add jupyter notebooks for wbsearchentities analysis	wikimedia/discovery/relevanceForge	master	+720 -0

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		• debt	T193701 Explore using user clicks data to tune Wikidata search parameters
		Resolved		EBernhardson	T211033 Analyze wbsearchentities AB test from nov/doc

Event Timeline

EBernhardson created this task.Dec 3 2018, 5:17 PM

EBernhardson triaged this task as Medium priority.

Restricted Application edited projects, added Discovery-Search; removed Discovery-Search (Current work). · View Herald TranscriptDec 3 2018, 5:17 PM

EBernhardson removed a project: Epic.Dec 3 2018, 5:17 PM

Addshore moved this task from incoming to monitoring on the Wikidata board.Dec 3 2018, 5:35 PM

EBernhardson claimed this task.Dec 6 2018, 6:14 PM

EBernhardson moved this task from needs triage to Current work on the Discovery-Search board.

EBernhardson edited projects, added Discovery-Search (Current work); removed Discovery-Search.

EBernhardson moved this task from Incoming to not in use - please delete on the Discovery-Search (Current work) board.Dec 6 2018, 9:56 PM

Report: F27421972

Summary: Improved Clicks@1 from 80% to 84% with no change to abandonment rate or number of characters typed.

Thanks for the report. It is odd that the number of characters didn't go down—as discussed elsewhere—but the change in clicks@1 vs clicks@2 is a nice clear step in a good direction.

• debt closed this task as Resolved.Dec 10 2018, 9:09 PM

Liuxinyu970226 unsubscribed.Dec 12 2018, 2:38 PM

EBernhardson mentioned this in T214515: Run wikidata entitiy autocomplete AB test in de, fr, es.Jan 23 2019, 8:56 PM

Change 486159 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[wikimedia/discovery/relevanceForge@master] Add jupyter notebooks for wbsearchentities analysis

https://gerrit.wikimedia.org/r/486159

gerritbot added a project: Patch-For-Review.Jan 23 2019, 9:03 PM

Change 486159 merged by jenkins-bot:
[wikimedia/discovery/relevanceForge@master] Add jupyter notebooks for wbsearchentities analysis

https://gerrit.wikimedia.org/r/486159