Page MenuHomePhabricator

[WE.1.2] Understanding editing pain points on iOS App
Closed, ResolvedPublic

Description

Please provide all the following information:

  • Context. Provide a short paragraph with some background context for your request, please include links to relevant material.
    • In order to determine which iOS app editing features to create during Q4-Q1, we would like design research support in understanding what accounts for the difference in constructive activation between mobile web and app users, and what are the most impactful ways to encourage constructive activation on the iOS app.
    • The iOS app plans to launch an “Edits tab” next fiscal, that will be an editing/community homepage for users. In support of WE1.2 for constructive activation during Q4, we want to prototype 2-3 ideas for what we can provide to new account holders in the Edits tab. Initially, we would like to conduct some design research to inform these ideas, and later, we may want to do some testing of these ideas or get some early feedback.
    • What we already know about the forthcoming Edits tab on iOS
      • We don’t plan to simply replicate Android’s tab, or the newcomer homepage on Web.
      • We have existing features for experienced editors that will have a home there, such as Watchlist and Add an image.
      • Adding access to user sandboxes has been prototyped in the past, and identified as a strong first opportunity for the tab.
      • Any new editing features we plan to prototype or develop should be available to all users regardless of edit count (in contrast to “Add an image” which is only available to users with 50+ edits).
      • Editing features should help new account holders “constructively activate”, or make their first main space edit 24-hours after creating an account, that isn’t reverted.
iOS app does not currently have a tab for EditingAndroid has "Edits" tabWeb has Newcomer Homepage
image.png (1×924 px, 255 KB)
image.png (1×900 px, 310 KB)
image.png (1×2 px, 646 KB)
  • Description.What is your request about?
    • What are the reasons that new account holders in the iOS app don’t start editing (“constructive activation”)?
    • What is the range of reasons that explain the comparatively low constructive activation rates on Mobile Apps? Our data shows that constructive activation rate on Mobile web is 17.6%; compared to 2.1% on Mobile Apps (based on data from March and April 2024; T360829: WE 1.2: Establish baseline for constructive activation).
    • The Contributor Strategy Brief identifies 4 key parts of the contributor cycle. Which part of the contributor cycle do new account holders on the iOS app struggle the most with? What are the most salient pain points for iOS app users?
      • 1) Discovering compelling opportunities to contribute. Potential pain points:
        • They wanted to edit but couldn’t find out where help was needed, or where they could make a difference
        • They aren’t aware of the many ways they can contribute
        • They don’t feel qualified to contribute.
        • They created the account for reading purposes and aren’t aware of editing
      • 2) Feeling equipped with the right tools and resources they need to seize these opportunities. Potential pain points:
        • They tried editing but found it too difficult or intimidating and never published
        • They wanted to learn about editing first before starting and looked for guidance
        • They tried editing but the user experience was too confusing, or not mobile-friendly
        • They're at an editathon with only their mobile device but finding it difficult to use the App's editor to make edits.
        • Their inconsistent internet connection makes it difficult to draft and edit content on Mobile.
      • 3) Understanding the impact of their work, and having a feedback loop. Potential pain points in the iOS app:
        • They don’t know if their edit will make a difference.
        • They don’t receive any feedback on their edits.
        • They aren’t sure if what they just published was useful to anyone
      • 4) Having clear next steps to act on after they take action, participating in what makes the movement so special – human connections. Potential pain points in the iOS app
        • Their first edit is reverted and they become discouraged.
        • They haven’t found a community or other editors to connect with.
        • They don’t know what to edit next.
    • How many users fall into each bucket? What are the most salient pain points for iOS app users? So that we know which opportunities will be most impactful to address.
    • Are there certain pain points that surface for specific language communities, or opportunities that certain language communities have that helps them be successful? We see variance in activation by language: the apps’ constructive rate ranges from 1.1% (zhwiki) to 6.8% (fawiki) (T360829).
    • We want to answer the above questions in early Q3. Once we know where the biggest opportunity is, we want to prototype 2-3 interventions that might help them activate during Q4.
  • Expected Deliverable. What is the ideal outcome or result of your request?
    • A report or slide deck presenting the findings by mid Q3, with a recommendation on the target pain points and audiences to focus on.
    • This research will inform the first feature to productionize in Q1 2025, however we would like to continue to use these research findings to inform multiple features for future development cycles.
  • Estimated Effort. Please provide an estimate of the amount of work needed to complete this task, if known.
    • We are open to methodologies and can provide some support. We have the ability to intercept users in context. For example, we could show a survey after account creation, or if the app user clicks edit button but starts abandoning, we could intercept with a one question survey, or link to a longer survey, or recruit in other ways. It may make sense to start with a short in-app survey, followed by interviews later in Q3 with design mockups that we can present to users for feedback.
    • Priority Please indicate a priority of your task and a small description of what it would unlock for you. We ask you to leave this task as “needs triage” since your request will go through a Backlog refinement process where our team will prioritize the work.

I need this task resolved in:

  • 1 month - 1.5 to 2 months, given the holidays.
  • 3 months.
  • 6 months.
  • Whenever you get to it :-)
  • Other. Do you have any other questions or comments ?
    • Timeline:
      • Early to mid Q3: research described above
      • Late Q3: Decide which prototypes to build
      • Q4: Prototyping and testing
      • Q1 2025: Production
      • Beyond Q1: Use research findings to explore more features that encourage editing on the mobile app.
    • Participants:
      • Logged-in users
      • Recently created an account on the app
      • Include: Those who did edit, those who haven’t edited
      • Languages (open to discussion with Design research on what would make a good sample to achieve the outcomes)
        • English
        • Chinese (below average activation rate)
        • Persian (above average activation)
      • Regions (open to discussion with Design research on what would make a good sample to achieve the outcomes). We have more data coming in January about edits and activation by language (https://phabricator.wikimedia.org/T382182), that could allow us to expand our audience if needed.
        • North America (lowest share of edits compared to active app installs and pageviews)
        • Middle East & North Africa (Above average edits compared to active app installs and pageviews)

For use by WMF Research team; please leave everything below as it is:

  1. Does the request serve one of the existing Research team's audiences? If yes, choose the primary audience. (1 of 4) Yes. Wikimedia Foundation.
  2. What is the type of work requested? Develop insights.
  3. What is the impact of responding to this request?
    • Support a technology or policy need of one or more WM projects
    • Advance the understanding of the WM projects.
    • Something else. If you choose this option, please explain briefly the impact below.

Details

Due Date
Apr 15 2025, 12:00 AM
Other Assignee
Bethany

Event Timeline

leila triaged this task as Medium priority.
leila removed a subscriber: Miriam.

@HNordeenWMF we reviewed your request and have prioritized the overall request that you have. We intend to start working on it this quarter. However, in our initial review we see that we may not be able to deliver it exactly as you have proposed in the task description (with the exact scope and timelines). @Mraish is leading this work on our end and he will be in touch with you in the coming week to explore options for re-scoping and/or updating timelines. If you need anything from me, please feel free to reach out.

leila renamed this task from [Request] Understanding editing pain points on iOS App to [Request] [WE.1.2] Understanding editing pain points on iOS App.Jan 17 2025, 10:47 PM
leila added projects: Design-Research, OKR-Work.
leila updated the task description. (Show Details)
Bethany changed the task status from Open to In Progress.Jan 28 2025, 8:48 PM
Bethany renamed this task from [Request] [WE.1.2] Understanding editing pain points on iOS App to [WE.1.2] Understanding editing pain points on iOS App.Jan 28 2025, 8:50 PM

Update: @MRaishWMF and I have been working with @HNordeenWMF and @scblr to clarify the project goals and scope the project. A research brief is written and still undergoing refinements (private link).

We continued to plan the project, finalized the research brief, and contacted our preferred Japanese research partner.

Weekly update: We submitted a BAR for the vendor who will support us with the Japanese interviews. We conducted an exploratory pilot, which was very informative.

HNordeenWMF closed subtask Restricted Task as Resolved.Feb 27 2025, 9:47 PM

Weekly update:
Conducted 2 pilots this week (possibly a 3rd on Sunday evening)
Supported Legal in writing contract terms for Japanese vendor
Summarized interviews for easy comparison analysis
Discussed first pilot with Haley, refined discussion guide
Received datapull of usernames from Shay.

Bethany reopened subtask Restricted Task as Open.Mar 7 2025, 5:23 PM

Weekly update:
Conducted the last (4th) pilot with an experienced editor
Created outreach materials, updated research materials
Weekly sync with Mike, Haley, and myself
Scheduled kickoff meeting with Japanese vendor for next week
Began sending recruitment messages for user interviews

Weekly update:
Mike and I conducted 3 interviews with our target population, which brings the total complete to 7 including pilots.
We onboarded Project Kobo, our research partner for the Japanese segment. I prepared all of the Japanese materials.
Mike began preparing materials for the Arabic segment.
Recruitment is ongoing (it has been a time-consuming process, set back by a couple of hiccups. We are also getting responses from users the data pull list who don't recall having made edits to WP, which disqualifies them for this study.)

Weekly update:
2 more interviews EN wiki participants
Revised materials for project scope pivot
Coordinated with JA vendor
Recruitment for JA segment (unfortunately, the number of JA participants with EmailUser from our data pull list is very low)
Began anaylsis

leila set Due Date to Apr 15 2025, 12:00 AM.Mar 21 2025, 5:35 PM

Thanks for the weekly update, Bethany. To that I add:

Bethany, Mike, Haley and I met yesterday to discuss the possibility of re-scoping this task in light of the fact that the upcoming fiscal year's draft plans for apps (work for which will start post July 1, 2025) will focus more heavily on readership work on apps (see WE.3.). We concluded that given that Mike and Bethany have made significant progress on this task, they will continue working on it. We also agreed on a few changes to make the study as helpful as possible for this FY's work as well as next's:

  1. We will reduce the scope of this study to en and ja (instead of the original plans to do it in en, ja, ar). This will allow the study to conclude on time and without spending more resources on this front.
  2. We will keep the primary focus of this study in understanding editing pain points, with a secondary focus on motivations for activation (what made them do the first edit?). This will be important because it allows us to give user-centered recommendations for how editing on apps should be improved if/when it's going to be prioritized in the future by the product teams.
  3. To the extend possible and without adding significant work, Bethany and Mike will include in their analyses and recommendations information about readership that may have been surfaced throughout the study.
HNordeenWMF closed subtask Restricted Task as Resolved.Apr 2 2025, 10:49 PM

I presented this research on April 16th, 2025. [private link]
Commons File here: https://commons.wikimedia.org/wiki/File:Reading_and_Editing_on_the_iOS_App_-_Apr2025.pdf

This project is complete. Thanks all!