Page MenuHomePhabricator

Run A/B Test of Wikipedia App Product Page for English users on Apple and Google App store
Open, Needs TriagePublic

Description

Hypothesis

We believe that showcasing the features valued by users based on reviews from the App Store and Google Play using modern visuals and benefit-focused messaging (e.g.: “ad free access to articles on millions of topics and languages”) that reflects the app’s current state

Will result in a higher installation rate and retention rate than the existing product page designs and copy

We will know this is true when our installation rate of the improved designs and copy exceeds the existing designs and copy on the app store.

Task

  • Prepare Figma screens in English for both platforms (phone and tablet) @TLessa-WMF
  • Launch A/B/C Test in Google Play Store and Apple App Store in English @TLessa-WMF
  • Check progress and determine if we should run the test longer @JTannerWMF
  • Provide recommendations on next steps @TLessa-WMF
  • Publish winning results

Designs

Control

Experiment B (Created by Derek+Taís)

Experiment C (Created by Robin)

Copy

Control:

Save and organize articles with Reading lists
Learn about nearby Places
Add Widgets to your home screen
Switch between articles with Tabs
Read comfortably in Dark mode
Explore new content on your customized feed
Search in 300+ languages
Edit articles with ease
Discuss articles with Talk pages
Stay updated with Notifications

Experiment:

Experimental Refreshed Copy for Android

Title (Short Description)

Wikipedia: Free knowledge without ads, always at your fingertips

Long Description (Body Copy)

The official Wikipedia app for Android helps you search, read, and contribute to the world’s largest free knowledge project — ad-free and free of charge, forever. Explore 60+ million articles in over 300 languages, wherever you are.

Why you’ll love this app

  1. Free, open, and ad-free

Wikipedia is created, edited, and verified by volunteers worldwide, supported by the nonprofit Wikimedia Foundation. No ads, no tracking — just knowledge you can trust.

  1. Search smarter, discover more

Find the information you need without distraction. Search across 300 languages, explore trending and breaking news, and save personalized recommendations to your reading list.

  1. Stay connected offline

Save articles to custom reading lists and access them offline — perfect for travel or learning without internet access.

  1. Test your knowledge

Try a daily trivia game to see how much you know and learn something new along the way.

  1. Explore nearby places

Discover articles about landmarks, cities, and places near you — or places you dream of visiting.

  1. Read your way

Choose from light, sepia, dark, or pure black themes. Adjust text size and switch to night mode for comfortable reading anywhere, anytime.

  1. Get insights into your activity

See your own reading and contribution activity on Wikipedia with personalized insights that help you understand and celebrate how you use the app.

  1. Stay organized with tabs

Open multiple articles in tabs and switch between them with ease, just like in a browser. Compare topics, keep your place, and dive deeper without losing track.

  1. Join the conversation

Read and reply to article discussion pages, and get notifications when discussions you follow are updated.

  1. Contribute on the go

Make micro-contributions with Suggested Edits or manage your Watchlist to help keep Wikipedia accurate and up to date.

Additional Information

Highlights Copy

Free knowledge made easy. Explore 60M+ articles ad-free in 300+ languages, wherever you are. Save articles for offline reading, get personalized recommendations, and track your learning. Play trivia, explore places, and stay organized with tabs — all free, forever, while protecting your privacy.

Due date

The test needs to be up and QAed by Dec 10

Event Timeline

English in this case means: India and South Africa - so we have to segment by country

HNordeenWMF renamed this task from Run A/B Test of Wikipedia Android App Product Page for English users on Apple and Google App store to Run A/B Test of Wikipedia App Product Page for English users on Apple and Google App store.Nov 25 2025, 6:13 PM

I asked this question via Slack but adding it here for documentation:

"I believe we are doing this test to take advantage of the increased flow of users due to the marketing banners, correct? One question I have is that the marketing team is A/B/C testing banners - doesn’t it make more sense if our install pages are stable, so we have fewer variables and can focus on comparing the impact of marketing messages on installs?"

Update: Jaz confirmed we wanted to do this in parallel. So here is the test plan, considering the current constraints of the platforms. The document was submitted for the team's review: https://docs.google.com/document/d/1juIrDZlNaTYcTEOJmTjKOaGd1yR1R-f1oKj3e9FaVoU/edit?usp=sharing

We will pick this up once Jaz returns to the office, and in the meantime, I will be working on the Figma screens (we don't have the tablet designs yet) to make sure designs are good to go while we align on the test plan.

TLessa-WMF updated the task description. (Show Details)
TLessa-WMF updated the task description. (Show Details)
TLessa-WMF updated the task description. (Show Details)
TLessa-WMF updated the task description. (Show Details)

Android (Google Play):
• draft ready
• missing updated Activity Tab screen - Sarah is working on it
• Note: since they only allow 8 screenshots, I removed 2 from the Light Blue variant - the one that mentioned editing and the one that mentioned notification. Since the new variant doesn't mention those 2 features, I think that this is a more equal comparison.
• Figma - Tablet: https://www.figma.com/design/QvsLu7llrctBWbAANlTHTG/iOS-%7C-Android---%3E-A-B-Test-Screenshots--App-Store-%7C-Google-Play-?node-id=3245-3184&t=d6DQoeYfl8IsbSGc-4 | Phone: https://www.figma.com/design/QvsLu7llrctBWbAANlTHTG/iOS-%7C-Android---%3E-A-B-Test-Screenshots--App-Store-%7C-Google-Play-?node-id=3283-9541&t=d6DQoeYfl8IsbSGc-4
• PNGs: https://drive.google.com/drive/folders/1BlvHgD6BDanrMs0_lm9bJetGnJm13qK8?usp=drive_link

@JTannerWMF the Android screens are now ready for your QA before I submit them to review - please go to the Google Play Console and take a look at the draft. Thanks!

Android test: Published and waiting for Google Play review.

Google Play test: published and active since Dec 11
App Store test: published and active since Dec 16