Run an A/B test to see whether or not defaultsort can improve user experience when search-as-you-type.
Context
The use of the defaultsort data in completion searches is believed to help in these cases:
- search for last-name first (previously enabled on the mongolian wikipedia: T327878)
- search for concepts that are often used with popular prefixes (search XYZ finds List of XYZ, see T386655)
The list of wikipedias where could enable this feature is (filtered on wikipedias that have more than 50% of the pages with a defaultsort that matches the pattern we expect and where it would increase recall on more than 20% of the pages).
| wiki | recall improvement |
| enwiki | 26% |
| dewiki | 28% |
| frwiki | 23% |
| eswiki | 21% |
| plwiki | 23% |
| fiwiki | 28% |
| nowiki | 23% |
| cswiki | 23% |
| hewiki | 28% |
| dawiki | 26% |
| simplewiki | 25% |
| bgwiki | 20% |
| etwiki | 21% |
| glwiki | 20% |
| afwiki | 33% |
| slwiki | 20% |
| lvwiki | 20% |
| vowiki | 47% |
| fywiki | 21% |
| mnwiki (x) | 20% |
| gvwiki | 25% |
| mtwiki | 21% |
| biwiki | 37% |
| iglwiki | 26% |
(x): already enabled in T327878
We could start an A/B test on 3 wikis first: enwiki, frwiki and hewiki.
AC:
- determine the set of wikis to test
- start an A/B test on these wikis
- analyse the results
- possibly enable this by default on these wikis and possibly infer if it can be enabled on more wikis