Tracking bug for graduating Hovercards out of Beta Features.
= Summary
HovercardsPage previews is a beta feature that has been tested by 10s of thousands of users across wikis. It shows a preview/summary of a page when a user hovers over a link. The user can move the mouse to dismiss the preview or can click on the link or the preview to continue onto the article.
This is a feature intended to improve the experience for any reader who normally would have clicked on a blue-link in wikipedia because they needed an overview (definition) of that entity. We know from preliminary survey results that ~1/3 of readers come to wikipedia for an overview, but once on site, when reading a page, the number is surely higher. The goal of hovercards is to:
# better serve the need when someone is looking for a definition,
- quicker
- less context switching
# not inconveniencing a user who wants a deeper dive into a subject
- only shows up after a pause
- is easily turned off
== Goal Visibility
Graduating hovercards on at least one wikipage previews on is committed externally here: [[ https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q4_Goals#Reading | https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q4_Goals#Reading ]]
https://www.mediawiki.org/wiki/Wikimedia_Audiences/2017-18_Q3_Goals#Readers
== Current Status
As of January, 2018, page previews are live on all wikipedias but German and English.
= Rationale
- well tested
- readers of catalan/greek wikipedia for whom this was turned on liked the feature 2:1
- relatively well liked by editors (recent english wiki RFC [[https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals)/Archive_131#Proposal:_Enable_Hovercards_by_default| shows mild preference against in it's current state]]
- leads to more learning! (quick glance at greek/catalan data shows that pageviews did not drop measurably, but the use of hovers was a significant additional action.
- arguably visually appealing
= Success Metrics
Our next step is to a/b test this on a smaller wiki. Our metric of success is that overall engagement pageviews + hovers increases significantly.
There is an issue to consider, which is that hovers can be accidental. To try and identify the impact of this issue and characterize the types of hovers to discount, we will be measuring hovers of a control group (without hovercards enabled) to identify what hovers look like and how many naturally would have triggered a hovercard if they were enabled.
= External Dependencies
- we might have some dependencies on the analytics team for ensuring that hovers appear where they should. this will depend on both the impact they have on pageviews and the scale of hoversThe results of A/B tests and qualitative testing on the feature can be found here: https://www.mediawiki.org/wiki/Page_Previews#Success_Metrics_and_Feature_Evaluation
Results from the most recent A/B test on English and German wikipedias are expected in early 2018
= Unknowns
Currently, we are investigating the following open questions
- We know most readers prefer them (~2:1)*. We need to confirm that for those who do not, it is either not a big deal or easy to turn them off.
- Are the images too big?
- Does this impact pageviews or fundraising?
- What does a positive hover look like? What % of hovers are accidental? How can we measure “engagement” for this feature?
= Product Plan
(in rough order)
- [- ]Generate rough timeline
- [- ]Rename
- [ - [-]Team instruments and familiarizes with code - soon
- [ - [-]Fix breaking bugs and move to RESTbase
- [ - [-]Rollout on small wiki
- [ - [-]Improve disable/enable workflow - in progress
- [ - [-]Qualitative analysis
- [ - [-]Quantitative analysis
- [ - [-]Image fixess
- [] Improving summary endpoint
== MVP
No new features: hovercards feature does not
- is scalable
- is net positive for users
- does not break existing workflows
- does not break anyone's experience
- can be measured
- is easily opted out of
- does not have misleading images
- does not interfere with fundraising
- community on any particular wiki is okay with it
== User Stories
As a reader of a wikipedia article looking at a blue link I am not familiar with, I want to learn what this [entity] is without losing my spot or waiting for another page to reload
== Metrics Implementation
For initial impact assessment, see: T131315
For tracking as an ongoing health metric, see "unknowns" section above
== Timeline Estimate
TBD
## Delivery Estimate
No sooner than June 30tWe are expecting to deploy to English and probably later.German wikipedias in Q3 of 2018 (Jan - Mar)