Based on the finalized design from T146663 and T139310, this A/B test will have a test group that will be shown additional information that will be relevant to individual results on the existing search results page. This additional information will be links and metadata of related pages with links to related categories and suggested language links that are similar to the individual search results. A control group will see the currently existing search results page.
This test is expected to last a week and will probably be run on English Wikipedia (desktop only) in June 2017. We'll use this ticket for the gerrit code check-ins and for turning the test on and off.
Test group users will see for each search result returned:
- 'explore similar' link and dropdown icon
- when the user hovers over the link of the search result, we will roll down a display below the result:
- a label 'Related pages' and
- up to 3 related page links containing metadata (as available):
- thumbnail image
- name of related pages
- metadata description
- up to 3 related page links containing metadata (as available):
- a label 'Categories' and
- a 3 column listing of links
- up to 3 links of related categories per column (for a maximum total display of 9 related categories)
- a 3 column listing of links
- a label 'Suggested languages'
- display of additional languages that the article is available in
- this uses the preferred browser language settings and the regional languages (using the universal language selector - user's IP region, browser preferred lang and the lang that the user has set as preferred within the Wikipedia account UI)
- when the user moves away from the new box, the links will roll back up
- if there aren't any articles in different languages, message that and add a link to translatewiki
Bucketing
Bucket testing logic generally is as follows:
- 1 in 1000 users are included in EventLogging (on enwiki that's about 2K full-text searchers, according to T163273)
- Of those 1 in 1000 users, 1 in 2 are included in the test
- Of those 1 in 2 users
- 50% go in a test group, labeled "serp_explore_similar"
- the other 50% of users will go in a control group, labeled "serp_no_similar_links"
- The remaining chunk (~1K) of the original bucketed ~2K users will get a NULL (the string null, or the MySQL null, we can detect either).
EventLogging
Eventlogging needs to capture:
- if the user hovered over the 'explore' links (to open the listings)
- if results for related pages were shown and how many
- if results for related categories were shown and how many
- if results for languages were shown and how many
- if the user clicked on a related page and which one
- if the user clicked on a related category and which one
- if the user clicked on a language link and which one
- we'd like to know if overall clicks go up or down for the search results