Background

• Release date: Dec 2, 2025
• 5 funnel day check: Dec 7, 2025 (Separate task)
• Display bug fixed: Dec 11, 2025
• 15 Days after bug fix - Dec 26, 2025
• 30 days - Jan 1. 2026 (separate task)

The Task

• Visualize and present the data in a way that is easily understandable to the team and helps answer the hypothesis

Requirements

• The data should be based on the metrics in the Epic

At 15 days:

Check metric-specific leading indicators: to answer hypothesis
Hypothesis: If we release Year in Review on Android and A/B test offering engaged users a reward to save a custom reading list, we will see a 1% increase in overall app retention rate amongst readers offered the reward compared to those who were not.

• KR 4.1 - 1% increase in overall app retention among readers who were offered the reading list (B) as compared to those who were not (A).
• KR 4.2 - 10% increase in Reading list engaged retention measured for the experiment group relative to control engaged
  • Reading List Engaged = visiting “saved”
  • Practical significance OK
  • Baseline from Discover: We saw a 64% increase in retention for the Variant Engaged uniques relative to Control Engaged uniques
• GR 4.3 - At least 80% of users rate the Year in Review reading list feature as satisfactory or neutral
• GR 4.4 - No more than 10% of survey respondents complain about privacy concerns related to the reading list (survey feedback)
• CR 4.5 - How did the conversion rate of the reading list compare to the app icon?
  • % of impressions of the reading list modal that clicked “Create”
  • vs % of slide impressions of the icon slide that turned on the icon