Page MenuHomePhabricator

Implement the "Exit the editor" survey
Open, In Progress, HighPublic8 Estimated Story Points

Description

Background goal

As part of T414710, we are going to run a survey to users with ≤100 cumulative edits who abandon mobile editing sessions after spending ≥2 seconds in either the mobile visual or source editor, we will discover patterns in the reasons behind this behavior and be able to decide what interventions we will prioritize to increase the mobile web edit completion rate.

This task is to implement the survey.

Survey timing

See T423923

Requirements

Target audience
  • Wikis: en.wiki
    • Start with en.wiki to avoid using time on translations during the initial test
  • Platform(s): Mobile web
  • Max edit count: ≤100 cumulative edits
  • Editing interface(s): mobile VE & mobile source
  • Account state: logged in and logged out
Triggering conditions

Show the survey when a user:

  • opens a mobile editing session
  • spends ≥2 seconds in either mobile visual editor or mobile source editor
  • leaves the editing experience without publishing a change
    • Where "leaves" in this context refers to someone either:
      • Explicitly tapping X within the source or visual editor
      • Tapping/swiping to navigate back to the previous web page they were viewing
Survey details

The survey will appear in a bottom sheet after leaving the editor.

image.png (3×4 px, 1 MB)

  • Title:
    • What stopped you from editing this article?
  • Description:
    • Your anonymous feedback will help us improve editing.
  • Radios with responses:
    • Just exploring
    • Not sure how to start editing
    • Worried about making a mistake
    • Interface felt too complicated
    • Tapped edit by accident
    • Ran into a technical issue
    • Something else
  • Button:
    • Submit feedback
  • Success Toast (if completed the survey)
    • Thank you for helping improve Wikipedia editing
Metadata
  • For every survey responses that is submitted, we will want to log the following information:
    • The editing interface someone was in when they abandoned
    • How long the editing interface took for them to load
    • Whether they did or did not make a change to the document before abandoning
    • Cumulative edit count
    • Edit session initiation method (section edit, full-page edit, etc.)
    • Account state (logged in / logged out)
    • Editing tools used (if using visual)
    • Edit session length
Implementation

This survey will be finally implemented using the existing feedback survey method that we're using in Edit Check decline surveys.

This approach will allow us to connect survey responses with edit session data (e.g. tools used, interaction patterns, load times), enabling deeper analysis beyond self-reported feedback.

NOTE: We will not finally use free responses (with an additional input) since it’s not allowed with this type of survey.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

An option we have is to use the existing feedback survey method that we're using in editcheck decline surveys, instead of QuickSurveys. This involves logging an event to VisualEditorFeatureUse, which can then be looked up.

Compared to QuickSurveys, this is much more limited in features. QuickSurveys can:

  • show many questions and group them together
  • ask freeform text questions
  • be configured and then automatically show to users based on various conditions (specific pages, account types, etc)

However, the only thing our intended survey means to do that's not supported by our existing feedback tool is a freeform text question. We only intend to show a single question, and we intend to manually trigger the survey based on a custom condition that QuickSurveys doesn't know about.

The useful bonus we'd get for not using QuickSurveys is that we'd be tied in to the existing edit session logging -- it'd be easy for whoever is analyzing these responses to look up things like "for users who said they ran into a technical issue, which tools did they attempt to use in that session? what was their load-time?"

Per offline discussion, we're going to move forward with the approach David outlined in T422931#11809005 provided, of course, @bmartinezcalvo does not see any blocking issues with doing so.

Accordingly, I've updated the task description.

This new approach makes sense since it will allow us to connect responses with edit session data. The only downside is that we won’t be able to capture additional context behind “Something else,” but anyway this new approach seems worth it given the benefits.

I've updated the proposal in the task description accordingly.

VPuffetMichel set the point value for this task to 8.Mon, Apr 13, 5:52 PM

Questions:

  • Do we want to show this to every user in the target group, or just a sample (e.g. 1 in 16)?
  • Do we want to potentially show this to the same user twice?

Questions:

  • Do we want to show this to every user in the target group, or just a sample (e.g. 1 in 16)?
  • Do we want to potentially show this to the same user twice?

Since we’re already targeting a specific group (≤100 edits, mobile, abandonment after ≥2s), I would suggest not sampling and showing it to all eligible users. This will help us gather enough data quickly in this initial phase.

For repeat exposure, I would recommend showing it only once per user (even if not responded) to avoid becoming intrusive.

Change #1271068 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/MobileFrontend@master] EditorOverlay: add a survey asking why people abandoned

https://gerrit.wikimedia.org/r/1271068

ppelberg changed the task status from Open to In Progress.Wed, Apr 15, 5:05 PM
ppelberg updated the task description. (Show Details)

Questions:

  • Do we want to show this to every user in the target group, or just a sample (e.g. 1 in 16)?
  • Do we want to potentially show this to the same user twice?

Since we’re already targeting a specific group (≤100 edits, mobile, abandonment after ≥2s), I would suggest not sampling and showing it to all eligible users. This will help us gather enough data quickly in this initial phase.

For repeat exposure, I would recommend showing it only once per user (even if not responded) to avoid becoming intrusive.

I think this is reasonable as well.

Some open questions that came to mind for me, sharing in case it helps folks implementing this:

  • Do we have any idea what the number of individual users who abandon an edit is within the group? (<100 edits)
    • If it's hundreds of thousands on a daily basis, you may want to sample
      • Or (again, if hundreds of thousands) maybe sample only for readers, and show to all logged in editors with <100 edits
  • Will the tool "know" for logged out users whether they've been exposed to the survey?
    • If not, again may want to sample for readers to reduce the risk of multiple exposure
  • Is it fairly easy to deploy/undeploy the survey?
    • E.g. if it takes days to remove but we're getting thousands of answers, either pay close attention to the response numbers and be ready to undeploy (if it goes on deployment trains, you may want to time around them, e.g. don't start on a Friday) or use sampling
  • Is the same survey configuration happening for both logged out and logged in users?
    • If readers are a much larger population of edit abandoners, they'll end up making way more of your response population (this is a problem with or without sampling; the only solution afaik is two separate configurations? Kinda per point 1, sub-point 2 - you'd likely want to sample readers in this case and display to all logged in editors with <100 edits)

Noting some slight deviations from the design comps:

  • Our OOUI form doesn't support error states (making the inputs red), so we just show the red warning message.
  • I updated the success message to "Thank you for helping to improve the editing experience!"
  • Toast on mobile is always white text on a black background

Noting some slight deviations from the design comps:

  • Our OOUI form doesn't support error states (making the inputs red), so we just show the red warning message.
  • I updated the success message to "Thank you for helping to improve the editing experience!"
  • Toast on mobile is always white text on a black background

@Esanders sounds good.

El T422931#11826424, @TAndic escribió:

For repeat exposure, I would recommend showing it only once per user (even if not responded) to avoid becoming intrusive.

I think this is reasonable as well.

Some open questions that came to mind for me, sharing in case it helps folks implementing this:

  • Do we have any idea what the number of individual users who abandon an edit is within the group? (<100 edits)
    • If it's hundreds of thousands on a daily basis, you may want to sample
      • Or (again, if hundreds of thousands) maybe sample only for readers, and show to all logged in editors with <100 edits
  • Will the tool "know" for logged out users whether they've been exposed to the survey?
    • If not, again may want to sample for readers to reduce the risk of multiple exposure
  • Is it fairly easy to deploy/undeploy the survey?
    • E.g. if it takes days to remove but we're getting thousands of answers, either pay close attention to the response numbers and be ready to undeploy (if it goes on deployment trains, you may want to time around them, e.g. don't start on a Friday) or use sampling
  • Is the same survey configuration happening for both logged out and logged in users?
    • If readers are a much larger population of edit abandoners, they'll end up making way more of your response population (this is a problem with or without sampling; the only solution afaik is two separate configurations? Kinda per point 1, sub-point 2 - you'd likely want to sample readers in this case and display to all logged in editors with <100 edits)

For the initial phase, I would propose a simple approach to avoid blocking the launch while still getting meaningful data:

  • Show the survey only once per logged-in user
  • Show the survey to 100% of logged-in users with ≤100 edits
  • Sample logged-out users to control volume and repeated exposure
  • If possible, we could monitor response volume daily and be ready to adjust the sampling rate or disable the survey if we reach a sufficient number of responses or if volume is higher than expected

Curious what others think @ppelberg @DLynch

Sample logged-out users to control volume and repeated exposure

I don't think there's any reason to treat logged out users differently. As currently implemented, they'll be shown the survey only once each just like logged in users.

Change #1271068 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontend@master] EditorOverlay: add a survey asking why people abandoned

https://gerrit.wikimedia.org/r/1271068

Per offline discussion, we've decided to show the survey to everyone who abandons an edit, regardless of how much time elapsed between the editing interface being ready and them seeking to abort.

We made this decision having confirmed that we will still be able to filter down survey responses based on how much time elapsed between the editing interface being ready and them seeking to abort.

With the above said, we will continue to scope the analysis we do to people who abandon/abort an editing session after ≥2 second elapses between the editing interface being ready and them seeking to abort.

ppelberg moved this task from Inbox to High Priority on the Editing QA board.

Priority
Set as high as QA is blocking start of survey, currently scheduled for Thursday, 23 April 2026.

Next steps

  • @bmartinezcalvo + @ppelberg + @EAkinloose to review implementation by EOD today on any production to using the following script in your browser console: mw.config.set( 'wgMFEnableAbandonSurvey', true );
  • @medelius to backport config to turn survey on tomorrow, 29 April 2026

I've reviewed the implemented survey and these are some design details to fix:

  1. Title text should use the heading style (18px size)
  2. Use @background-color-base in the bottom sheet
  3. The X button is not well aligned on top with the bottom sheet's title
  4. Bottom sheet's paddings (right, left, top, bottom) should be 16px (now it's 12px)
  5. The "Submit feedback" button is not visible
Captura de pantalla 2026-04-28 a las 19.19.55.png (667×375 px, 60 KB)
image.png (933×428 px, 82 KB)
ImplementationDesign (expected behavior)

Next steps

  • @bmartinezcalvo + @ppelberg + @EAkinloose to review implementation by EOD today on any production to using the following script in your browser console: mw.config.set( 'wgMFEnableAbandonSurvey', true );

I've not yet been successful in getting the survey to activate [1] and I think the reviews @bmartinezcalvo did and @EAkinloose will be doing are sufficient.

Taken all together: once the edits @bmartinezcalvo specified in T422931#11867284 are implemented, I think the survey can begin.[1]


  1. I tried while logged out b/c I've made ≥100 edits at en.wiki with both my personal and staff accounts.
  2. This, of course, assumes Esther does not find any blocking issues

Peter, it looks like it didn't ride the train last Monday. Not sure if there's a reason I'm missing there? There was no train last week, so you wouldn't see it on enwiki until Thursday, if that's where you're trying.

Change #1278687 had a related patch set uploaded (by Medelius; author: Medelius):

[mediawiki/extensions/MobileFrontend@master] Abandon editor survey: UI updates

https://gerrit.wikimedia.org/r/1278687

Test wiki created on Patch demo by CMedelius-WMF using patch(es) linked to this task:
https://2b5ea87048.catalyst.wmcloud.org/w/

I've reviewed the implemented survey and these are some design details to fix:

  1. Title text should use the heading style (18px size)
  2. Use @background-color-base in the bottom sheet
  3. The X button is not well aligned on top with the bottom sheet's title
  4. Bottom sheet's paddings (right, left, top, bottom) should be 16px (now it's 12px)
  5. The "Submit feedback" button is not visible
Captura de pantalla 2026-04-28 a las 19.19.55.png (667×375 px, 60 KB)
image.png (933×428 px, 82 KB)
ImplementationDesign (expected behavior)

Bárbara, could you check out the patch demo I created? I resolved 1-4. For 5, I made it so the button can be scrolled to if it doesn't fit already. I could also keep the button fixed at the bottom and make the survey questions scrollable instead? Let me know.

Test wiki on Patch demo by CMedelius-WMF using patch(es) linked to this task was deleted:

https://2b5ea87048.catalyst.wmcloud.org/w/

@medelius I've reviewed the fixed implementation in the patch and it looks good. Just the text style of the title in the bottom sheet should use the line-height-large (28px).

@medelius I've reviewed the fixed implementation in the patch and it looks good. Just the text style of the title in the bottom sheet should use the line-height-large (28px).

@medelius can create a new ticket to track the line height. Unblocking this for now

Priority
Set as high as QA is blocking start of survey, currently scheduled for Thursday, 23 April 2026.

Sorry for the delay but I wanted to ask: Can we start including a bit more detail when something moves to high priority? For instance:

What environment it can be found in
What is most pressing to validate (if necessary - on this ticket it's clear that "submit" is important)
Reasoning for why this would block a release (again, if we can't close the survey out and someone is stuck forever that's bad)

That would help QS take more immediate action when we see something move into High priority.

Next steps

  • @bmartinezcalvo + @ppelberg + @EAkinloose to review implementation by EOD today on any production to using the following script in your browser console: mw.config.set( 'wgMFEnableAbandonSurvey', true );

✅ Done

  • @medelius to backport config to turn survey on tomorrow, 29 April 2026

#TODO

@medelius I've reviewed the fixed implementation in the patch and it looks good. Just the text style of the title in the bottom sheet should use the line-height-large (28px).

As decided during the planning meeting, I've created this separate task T424874 to fix this in the future sprint. So we can launch the survey as it is.

Change #1278687 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontend@master] Abandon editor survey: UI updates

https://gerrit.wikimedia.org/r/1278687

Change #1279448 had a related patch set uploaded (by Medelius; author: Medelius):

[mediawiki/extensions/MobileFrontend@wmf/1.46.0-wmf.26] Abandon editor survey: UI updates

https://gerrit.wikimedia.org/r/1279448

Change #1279448 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontend@wmf/1.46.0-wmf.26] Abandon editor survey: UI updates

https://gerrit.wikimedia.org/r/1279448

Mentioned in SAL (#wikimedia-operations) [2026-04-29T20:32:50Z] <dancy@deploy1003> Started scap sync-world: Backport for [[gerrit:1277569|Enable mobile editor abandonment survey on enwiki (T423923)]], [[gerrit:1279448|Abandon editor survey: UI updates (T422931)]]

Mentioned in SAL (#wikimedia-operations) [2026-04-29T20:34:48Z] <dancy@deploy1003> dancy, caro, kemayo: Backport for [[gerrit:1277569|Enable mobile editor abandonment survey on enwiki (T423923)]], [[gerrit:1279448|Abandon editor survey: UI updates (T422931)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-04-29T20:44:33Z] <dancy@deploy1003> Finished scap sync-world: Backport for [[gerrit:1277569|Enable mobile editor abandonment survey on enwiki (T423923)]], [[gerrit:1279448|Abandon editor survey: UI updates (T422931)]] (duration: 11m 43s)

Change #1280519 had a related patch set uploaded (by Medelius; author: Medelius):

[mediawiki/extensions/MobileFrontend@master] Abandon the editor survey: update edit count restriction

https://gerrit.wikimedia.org/r/1280519

Change #1280519 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontend@master] Abandon the editor survey: update edit count restriction

https://gerrit.wikimedia.org/r/1280519

Change #1280552 had a related patch set uploaded (by Medelius; author: Medelius):

[mediawiki/extensions/MobileFrontend@wmf/1.46.0-wmf.26] Abandon the editor survey: update edit count restriction

https://gerrit.wikimedia.org/r/1280552

Change #1280552 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontend@wmf/1.46.0-wmf.26] Abandon the editor survey: update edit count restriction

https://gerrit.wikimedia.org/r/1280552

Mentioned in SAL (#wikimedia-operations) [2026-04-30T20:42:25Z] <jhuneidi@deploy1003> Started scap sync-world: Backport for [[gerrit:1280516|Suggestion Mode controlled experiment: limit exposure to newcomers (T422736)]], [[gerrit:1280552|Abandon the editor survey: update edit count restriction (T422931)]]

Mentioned in SAL (#wikimedia-operations) [2026-04-30T20:44:05Z] <jhuneidi@deploy1003> caro, jhuneidi: Backport for [[gerrit:1280516|Suggestion Mode controlled experiment: limit exposure to newcomers (T422736)]], [[gerrit:1280552|Abandon the editor survey: update edit count restriction (T422931)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-04-30T20:50:59Z] <jhuneidi@deploy1003> Finished scap sync-world: Backport for [[gerrit:1280516|Suggestion Mode controlled experiment: limit exposure to newcomers (T422736)]], [[gerrit:1280552|Abandon the editor survey: update edit count restriction (T422931)]] (duration: 08m 33s)