Page MenuHomePhabricator

Create Experimentation Plan for Sharable Reading List
Closed, ResolvedPublic

Description

Background
The Android team is working on a project to share reading lists. Full details about implementation can be found in T313269.

The team set the following Objectives and Key Results
Objective 1: Target growth readers find and share content that is interesting and relevant to them
Quant Target Region: Africa (Egypt, Algeria, Morocco, Kenya, DR Congo, Angola, Nigeria and Ghana) and South Asia (India, Bangladesh, Pakistan, Sri Lanka, and Nepal)
KR 1.1: At least 20% of initial users will engage with the feature again within 30 days after initial engagement
KR 1.2: Increase pageviews in target growth markets by 5% within 90 days of release
KR 1.3: Increase app downloads from target growth markets six months after release at a rate higher than the same time period the previous two years
Qual Target Audiences: Usability testing and design research must be conducted with low vision and low bandwidth users. Usability testing and design research must be conducted in target regions and top 5 languages of each region with diversity in age, screen size, low tech and app experience, and with no more than 40% of male participants.

Hypothesis & Assumptions: We believe that by having shareable reading lists that can be accessed by app only, where Android is the leading OS, it will encourage users receiving the reading list to download the app if they do not have it currently. Further, we believe the act of sharing the list will prompt users to read an article when their motivation to read Wikipedia may have otherwise been different.

In order to test our Hypothesis and Assumptions and measure the KRs above, our team will need to have a measurement and experimentation strategy. See Growth team Plan as an example.

The Task
Analyst and PM collaborate to create and publicly post our measurement and experimentation plan.

Must Do

  • Work with Designer to write out Research questions taking into consideration our hypothesis and assumptions
  • Create leading indicators and plans of action
  • Work with Engineering to determine necessary instrumentation
  • Share plan with Legal and CRS

Event Timeline

JTannerWMF renamed this task from [placeholder] Experimentation Plan to Create Experimentation Plan for Sharable Reading List.Aug 8 2022, 3:42 PM
JTannerWMF reassigned this task from JTannerWMF to SNowick_WMF.
JTannerWMF triaged this task as Medium priority.
JTannerWMF updated the task description. (Show Details)
JTannerWMF subscribed.

What we are tracking:

  • Referral links from installs generated by shared Reading Lists

https://meta.wikimedia.org/wiki/Schema:MobileWikiAppInstallReferrer
(This should also be tracked in Google Play Store which we can collect manually.)

See Plan Deck for event metric details.

JTannerWMF raised the priority of this task from Medium to High.Nov 2 2022, 8:57 PM

Lets update this with the notes from today's meeting

Baseline Data for Test Groups: Data

90 Day Date Range: 2022-09-01 - 2022-11-30, stats are 30 Day event/unique counts averaged from 90 days of data.

@JTannerWMF Note OKR 1.3 new install data will be collected from internal data (MobileWikiAppInstallReferrer) and Play Store data.