Page MenuHomePhabricator

Analyze wbsearchentities AB test from nov/doc
Closed, ResolvedPublic

Description

From the work in T209402 we started an AB test last thursday. It should be turned off on dec 6th and the results reviewed. Metrics:

  • % of users satisfied after n characters
  • click position
  • session abandonment

Event Timeline

EBernhardson created this task.

Report: F27421972

Summary: Improved Clicks@1 from 80% to 84% with no change to abandonment rate or number of characters typed.

Thanks for the report. It is odd that the number of characters didn't go down—as discussed elsewhere—but the change in clicks@1 vs clicks@2 is a nice clear step in a good direction.

Change 486159 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[wikimedia/discovery/relevanceForge@master] Add jupyter notebooks for wbsearchentities analysis

https://gerrit.wikimedia.org/r/486159

Change 486159 merged by jenkins-bot:
[wikimedia/discovery/relevanceForge@master] Add jupyter notebooks for wbsearchentities analysis

https://gerrit.wikimedia.org/r/486159