Background

In May 2021 the Android team built an MVP for Image recommendations. The goal of the MVP was to determine if it would be a good* task for newcomers, especially on mobile. The results were generally positive resulting in the Growth team introducing the feature on Web (Desktop and Mobile). It has had mostly positive results and the community is acquainted with the feature giving us the confidence to bring the feature back to the app with the ability to actually post photos.

We want to measure:

Metric specific Leading Indicators: Indicators to be captured after 15 days
Image rejection rate does not exceed 29%
Edit over-acceptance rate (never skip or reject recommended images) does not exceed 35%
Task completion rate is not below 30%
Revert rate does not exceed 18%

Validation

• KR 1.1: 2000 articles have images in a 30 day period
• KR 1.2: Average at least 8 edits per day per unique user
• KR 1.3: 15% of eligible Suggested Editors try image recommendations task
  • by app_install_id, just by username in mediawiki_history)
• KR 1.4: 70% of those users complete the task again on a separate day in a 15 day period
• KR 1.5: Reject and Accept rate does not deviate from Mobile Web or MVP by more than 10% (redundant metrics from
• KR 1.6: DAU of Suggested Edits increase overall

Guardrails

• KR 1.1: Feature does not worsen gender or geographic bias*
• KR 1.2: Less than 5% of users report NSFW or offensive content
• KR 1.3: Users spend at least 10s evaluating a task before publishing it
• KR 1.4: Bounce rate does not exceed 50%
  • Bounce rate defined as users that click Yes then abandon the flow before publishing
• KR 1.5: At least a 35% task completion rate
  • Defined as users that click on Add an image as a task, and actually clicks Yes, No or Not sure (interact with the feature)
• KR 1.6: Revert rate does not exceed 18%

Curiosities (nice to have)
KR 1.1: Do these numbers differ by language or user tenure?
KR 1.2: If this is a user’s first suggested edit, do they go on to try others?
KR 1.4 At what point in workflow are most frequent dropoff events?
KR 1.3: Feature perception by geographically underrepresented groups on large language wikis

Target Quant Regions and Languages

• Spanish Wikipedia
• Portuguese Wikipedia
• Persian Wikipedia
• Hindi Wikipedia

Target Qualitative Audience

• Spanish and Portuguese speakers in LATAM and Caribbean countries
• Hindi speakers in India
• Persian speakers across diaspora
• No more than 40% of our testers should identify as male

Must Haves for V1

• Entry point should reside in Suggested Edits screen and within an article
• Image Recommendations task should exist immediately below Article Descriptions
• There should be an indicator it is a new task to users that have completed at least one suggested edit and the new indicator should go away the second time the visit the screen
• Users should have an option to add a caption on the screen but it should not be prompted
• There must be a way for users to report issues with the feature, it should take them to the support email
• Users should have an entry point in the task to the FAQ page
• There should be some sort of way to ensure users aren’t continuously hitting yes in a short (less than 5s) span of time to guard against bad faith edits
• There must be onboarding to the feature
• Users should be able to access full article and image metadata
• Do not allow users to skip without providing some feedback (Yes, No, Not Sure)
• Allow users to zoom/pinch on image
• Considerations for images running out or task not being available in certain languages

Nice to Haves

• Users able to filter by topics
• Subtle positive reinforcement (display in darkmode)

Potentially out of scope

• Section Images

Things to consider

• How to improve captions from the Growth team's iteration
• If there should be additional requirements for who can do the task

User Stories

• As a Wikipedia Android app user with a small screen and inconsistent internet, I would like to evaluate images and determine if they should go into an article, to contribute to Wikipedia articles that are in need of more content.

• When I am using the Android app, I want to be able to add images with one hand to many articles on my mobile device, so that I can be productive while riding the bus in Bogotà, and listening to music.
• When I am reading an article about Allameh Jafari Bridge in Iran, I want to learn that I can add an image to the Persian language article, so that I can enhance articles that I find interesting.
• When I try to add an image to an article and fail, I want to learn how to do this task, so that I can build my confidence before going on to adding image that I want as my intended edit.
• BONUS: When I am recommended an image for an article in Bengali and I don’t believe the image is a good fit and I am physically next to the subject of the article, I want to be made aware there is a Commons app, so I can upload the appropriate image using the Commons App