Page MenuHomePhabricator

Explore Similar Language Link A/B test
Closed, ResolvedPublic

Description

We'll be doing another test to investigate if and how many users click into the language link to see if the article is also written in a language (other than English) and then to see if the user wants to click through to that new article to read.

Write up is here: https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_Improvements/explore_similar#Explore_similar_-_for_languages_in_search_results

It'll look something like this:

Event Timeline

debt created this task.Sep 11 2017, 11:33 PM
Restricted Application edited projects, added Discovery-Search; removed Discovery-Search (Current work). · View Herald TranscriptSep 11 2017, 11:33 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
debt added a comment.Sep 12 2017, 5:44 PM

This new test code will ride the train this week and be turned on Thursday, Sep 14: https://gerrit.wikimedia.org/r/#/c/376508/ and https://gerrit.wikimedia.org/r/#/c/376509/

Test has been turned back off. Ready for analysis.

debt closed this task as Resolved.Sep 22 2017, 2:23 PM

The analysis is done for the running of this test and unfortunately, we only had one clickthrough to an article written in a different language (which was displayed in the new language links) as the report documents. We will not be going forward with this feature.

However, if a user wants to have the language link script added to their logged-in account, please follow the instructions here: https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_Improvements/explore_similar#Explore_similar_-_for_languages_in_search_results.

The full explore similar script (displays related articles, categories and language links) can also be enabled for logged-in users, see this link for instructions: https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_Improvements/self-guided_testing#Explore_similar

debt added a comment.Sep 22 2017, 7:35 PM

Breakdown of events during this 7 day test cycle on English Wikipedia, taking into account that these numbers are only a bucketed sampling of all users that saw a SERP page during this test:

Language link was clicked on to display list of the same article but written in a different languageclickthrough to same article written in a different language%
7010.14%
Total SERPs control groupSERP w/sis project sidebar%clickthrough to sis project%
1,6451,21173.62%90.74%
Total SERPs test groupSERP w/sis project sidebar%clickthrough to sis project%
1,5941,19875.16%30.25%
Total SERPs control groupclickthrough to article%
1,64553232.34%
Total SERPs test groupclickthrough to article%
1,59443327.16%

Data Summary graph: