Page MenuHomePhabricator

[SPIKE] Determine instrumentation requirements for A/B test section snippets
Open, HighPublic

Description

Background

Search engines have been experimenting with surfacing subsection links and snippets for structured pages. Their goals are to make it easier for users to find the most relevant section of a page for their search. We would like to test whether allowing our content to be displayed as snippets affects readership metrics

We would like to perform an A/B test across multiple languages. The A/B test will show the snippets on a certain set of pages (potentially 50%). We will study the results of the test to ensure that there are no other effects on pageviews or other important readership metrics.
Languages for the test:

  • English
  • Spanish
  • French
  • Portuguese
  • Arabic
  • Hindi
  • Bengali
  • Indonesian
  • Japanese
  • Russian

Hypotheses

Pages that are in the test group (which are marked to display snippets within Google search results) will have more pageviews than the control group from external search referrals for all languages studied
Overall pageviews in the test group will not be affected negatively for all languages studied

Questions to answer