Background

As the Foundation focuses on creating a multigenerational platform, the mobile apps are being leveraged as a flagship experience for deep readers and knowledge lovers.

Over the past five years the Android app focused on building features for editors. Due to this strategic shift of transforming the app to a place for deep readers and donors, we will expand the Edit tab which is exclusive to edit insights, to include insights for readers and donors.
Through last year’s experiments with Year-in-Review, an activity tab on iOS and Contributions Dashboard, we learned that many of our users appreciate seeing insights about their behavior. It was also clear based on user feedback that users want to have control over what is displayed and ensure privacy is honored, which is why with the exception of editing data that is already public, we will leverage local data for this implementation.

Hypothesis

If we expand the existing Edit tab on Android into a personalized activity hub that includes insights into reading and non-editing participation, we’ll see a 5% increase in multi-day engagement with the tab compared to the original version.

Our hypothesis rolls up to the 3.3 KR of engaging core audiences.

The full key result reads:

By the end of Q2, demonstrate a practically significant increase in logged-in reader retention, as measured through A/B testing of one feature per platform.

Additional Context:
This KR will focus on improving the reading and learning experience for existing and experienced readers, with the goal of retaining our current audience and deepening their connection to the site so they can learn more, as well as be ready and open to take paths towards donation and editing. Work here will focus on improving the reading experience on the web and apps (readability improvements, better navigation and discovery), as well as building out and iterating on our curation and personalization offerings (Reading lists, personalized suggestions, user and article history, etc)

How will we know we are successful?

Validation

• Primary KR 1.1 – 5% higher multi-day visits of activity tab among logged-in users who have the new activity tab vs those that have the edit tab
• KR 1.2 – A statistically significant higher retention rate of experiment group compared to control group

Guardrails

• GR 2.1 – No more than 25% of users turns feature completely off
• GR 2.2 – At least 80% of qualitative feedback received is positive or neutral

Curiosities

• CR 3.1 –What % of people switch their new tab experience to something other than the default?
• CR 3.2 –Which modules are people turning off and at what rate?
• CR 3.3 – How does KR 1 and 2 change when isolating users that are logged in with 0 edits?
• CR 3.4- How many new account creations go on to make an edit after seeing the activity tab?
• CR 3.5- How many new accounts go on to engage with non editing participation features after seeing the activity tab?

• Non editing participation is defined as adding articles to their reading list or playing a game

Requirements

Must have:

• Experiment will be ran as an A/B Test
  • Control: Existing Edit Tab
  • Experiment: New Activity Tab
• Users must be able to hide elements of the module
• It must be clear on the page where users can turn off modules that we are using local data for non editing data
• Access to an info page that leads to mediawiki
• It must be clear where local data is being used
• Modules are clickable and when empty includes CTAs
• Update Navigation bar to read Activity instead of edit
• Show red dot indicator so users notice there has been a change
• Require user to have an account to see activity
• The order of the modules in the activity tab should be as follows:
  • Time Spent reading this week
  • Monthly Reading Insights
    • Articles read
    • Articles Saved
    • Categories Read
  • Monthly Editing Insights
    • Most viewed article
    • Contributions
  • All time Impact
    • Total Edits
    • Thanks Received
    • Best Edit Streak
    • Last time edited
    • Article edited with most views
    • All Time Views on articles edited
  • Game Stats
    • Games Played
    • Current Streak
    • Average Score
  • Last in app donation (hidden by default)
  • Timeline of behavior
    • Reading history
    • Editing history

Nice to have

• Ability to search page
• Reminders for reading and editing goals
• Charts to visualize activities

Designs

Figma

Core Screens

Test Languages

German & Japanese

Testplan

https://docs.google.com/document/d/13cBItD76Tbgp86Lj3z-PRx-b6ltLhgyZMvDkSWgS2Ow/edit?usp=sharing

Reference Documents

• Shallow Dive Deck