[GOAL] Graduate Page Previews feature (Popups extension) out of Beta Feature
Closed, ResolvedPublic

Description

Summary

Page previews is a beta feature that has been tested by 10s of thousands of users across wikis. It shows a preview/summary of a page when a user hovers over a link. The user can move the mouse to dismiss the preview or can click on the link or the preview to continue onto the article.

This is a feature intended to improve the experience for any reader who normally would have clicked on a blue-link in wikipedia because they needed an overview (definition) of that entity. We know from preliminary survey results that ~1/3 of readers come to wikipedia for an overview, but once on site, when reading a page, the number is surely higher. The goal of hovercards is to:

  1. better serve the need when someone is looking for a definition,
    • quicker
    • less context switching
  2. not inconveniencing a user who wants a deeper dive into a subject
    • only shows up after a pause
    • is easily turned off

Goal Visibility

Graduating page previews on is committed externally here: https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q4_Goals#Reading
https://www.mediawiki.org/wiki/Wikimedia_Audiences/2017-18_Q3_Goals#Readers

Current Status

As of January, 2018, page previews are live on all wikipedias but German and English.

Rationale

  • well tested
  • readers of catalan/greek wikipedia for whom this was turned on liked the feature 2:1
  • relatively well liked by editors (recent english wiki RFC shows mild preference against in it's current state
  • leads to more learning! (quick glance at greek/catalan data shows that pageviews did not drop measurably, but the use of hovers was a significant additional action.
  • arguably visually appealing

Success Metrics

Our next step is to a/b test this on a smaller wiki. Our metric of success is that overall engagement pageviews + hovers increases significantly.

There is an issue to consider, which is that hovers can be accidental. To try and identify the impact of this issue and characterize the types of hovers to discount, we will be measuring hovers of a control group (without hovercards enabled) to identify what hovers look like and how many naturally would have triggered a hovercard if they were enabled.

The results of A/B tests and qualitative testing on the feature can be found here: https://www.mediawiki.org/wiki/Page_Previews#Success_Metrics_and_Feature_Evaluation

Results from the most recent A/B test on English and German wikipedias are expected in early 2018

Product Plan

(in rough order)

  • Generate rough timeline
  • Rename
  • Team instruments and familiarizes with code - soon
  • Fix breaking bugs and move to RESTbase
  • Rollout on small wiki
  • Improve disable/enable workflow - in progress
  • Qualitative analysis
  • Quantitative analysis
  • Image fixes
  • Deploying improved summary endpoint
  • Publishing A/B test results
  • Final community consultation prior to deployment

Remaining work tracked under T154635: [EPIC] Deploy page previews to English and German Wikipedia

MVP

No new features: hovercards feature does not

  • is scalable
  • is net positive for users
  • does not break existing workflows
  • does not break anyone's experience
  • can be measured
  • is easily opted out of
  • does not have misleading images
  • does not interfere with fundraising
  • community on any particular wiki is okay with it

User Stories

As a reader of a wikipedia article looking at a blue link I am not familiar with, I want to learn what this [entity] is without losing my spot or waiting for another page to reload

Delivery Estimate

We are expecting to deploy to English and German wikipedias in Q3 of 2018 (Jan - Mar)

Details

Reference
bz68860

Related Objects

StatusAssignedTask
ResolvedDereckson
ResolvedJdlrobson
ResolvedNone
ResolvedGilles
Resolvedovasileva
DeclinedNone
ResolvedKrinkle
ResolvedNirzar
DeclinedNone
Resolvedovasileva
DeclinedNirzar
ResolvedJKatzWMF
Resolveddr0ptp4kt
ResolvedJdlrobson
Resolvedphuedx
ResolvedJdlrobson
InvalidNone
Resolvedovasileva
Resolvedovasileva
ResolvedJdlrobson
ResolvedNone
ResolvedJdlrobson
ResolvedJhernandez
DuplicateNone
ResolvedPrtksxna
DeclinedNirzar
Resolvedphuedx
Resolvedphuedx
DuplicateNone
Resolveddr0ptp4kt
DuplicateNone
DuplicateNone
Resolvedovasileva
ResolvedJdlrobson
Resolvedphuedx
ResolvedABorbaWMF
Resolvedovasileva
Resolvedovasileva
Resolvedovasileva
DeclinedNone
There are a very large number of changes, so older changes are hidden. Show Older Changes

Took a spin through with the Web team and identified candidates for Q2 Milestone

rugk added a comment.Jan 31 2017, 5:18 PM

So how can I turn it on (without being logged in)?

Restricted Application added a subscriber: jeblad. · View Herald TranscriptAug 25 2017, 8:44 PM
jeblad removed a subscriber: jeblad.Aug 25 2017, 10:18 PM
ovasileva moved this task from Backlog to Next Up on the Page-Previews board.Dec 12 2017, 3:51 PM
ovasileva renamed this task from [GOAL] Graduate Hovercards feature (Popups extension) out of Beta Feature to [GOAL] Graduate Page Previews feature (Popups extension) out of Beta Feature.Jan 11 2018, 9:52 PM
ovasileva updated the task description. (Show Details)
ovasileva updated the task description. (Show Details)
ovasileva updated the task description. (Show Details)
ovasileva updated the task description. (Show Details)
ovasileva updated the task description. (Show Details)Jan 11 2018, 11:55 PM
ovasileva raised the priority of this task from Normal to High.Jan 23 2018, 1:34 AM
phuedx awarded a token.Jun 5 2018, 5:06 AM