Page MenuHomePhabricator

Newcomer tasks: increased flexibility at post-edit dialog
Closed, ResolvedPublic

Description

Background

In our analysis of the "add a link" funnel, we saw that in a user's first session through "add a link", only 28% of desktop users and 11% of mobile users choose to do another task from the "post-edit dialog". The most common action was to close the dialog by clicking or tapping outside of it (or clicking the "X" on desktop).

We think that many users may be closing the dialog because they want to look at the edited article to confirm/admire the changes they made to it. Once they close the dialog, they can't open it back up, and so have lost the opportunity to edit the next article.

image.png (876×1 px, 83 KB)

Hypothesis

If we affirmatively design for that "admiring" behavior by encouraging it, but then still allow users to return to the dialog to choose a next article, they will both admire their work and edit another article.

Proposed design B: Separate success message & next edit

This proposal firstly decouples the post-edit message from the next task. The second main change is that the next suggestion takes the form of a collapsible drawer akin to the ‘inspector’ component in structured tasks.

  • Pros
    • Removes likelihood of accidentally closing the dialog
    • Collapsed version of next edit inspector doesn’t obscure any article content.
    • Removing the modal and separating the success is more in consistent with “normal” edits (where only the success message appears as a toast)
    • Separating the success message enables more flexibility with other Positive reinforcement experiments related to “celebration” vs specific recommended next actions.
  • Cons
    • Separating the two makes the review my edit aspect less overt as a an action for users
    • Physical removal of the “close” button makes it less obvious how one could minimise the view to see the article they have just edited
  • Other notes on design
    • Users will not be able to remove this next edit UI unless they refresh the page.
    • The random/reload new suggestion action will be replaced with navigation buttons to flip through solutions as per T302335
    • On Desktop, the next suggestion dialog is transformed into the same bottom drawer as the Desktop Add image card. The success message will be more separated at the top of the screen similar to other other Visual editor success messages.
Additional animation/transition details

Mobile

  1. Submit edit
  2. Success/Completed message appears (fade up and animate in from bottom of screen) [200ms]
  3. Message remains in place for [400ms]
  4. Next edit animates in from bottom, “pushing” up the message toast. [400ms]
  5. If the message toast is not tapped/clicked on by the user to dismiss it immediately, it will remain in place for [5s]
  6. Message toast fades out in [200ms] duration – either after the normal period, or if the user selects it to dismiss it.
Mock showing sequence
image.png (1×1 px, 331 KB)
Recording of prototype

Mobile keyframes start in this Figma frame

Desktop animation

  1. Submit edit
  2. Success/Completed message appears (fade in and animate downward from top of screen) [400ms] simultaneous to the page transitioning from the Edit to Read mode.
  3. Message remains in place for [400ms]
  4. Next edit animates in from bottom [400ms]
  5. If the message toast is not tapped/clicked on by the user to dismiss it immediately, it will remain in place for [5s]
  6. Message toast fades out in [200ms] duration – either after the normal period, or if the user selects it to dismiss it.
{F34970426}

Desktop animation available here | Keyframes start in this Figma frame

Apply changes to
  • Mobile and Desktop Post-edit UI on structured tasks (Add links and Add image) when there has been an edit made
  • Mobile and Desktop Post-edit UI on structured tasks (Add links and Add image) when there has been NO edit made (e.g., link is rejected)
  • Mobile and Desktop Post-edit UI on the Add image task when the quality gate quote is reached (with and without an edit)

[x ] Mobile and Desktop Post-edit UI on all other suggested edits tasks (e.g., copyedit, add references, etc)

Related changes
  • Instrumentation
Use caseExisting eventNew (proposed) eventNotes
Toast messageNone (message is part of dialog)postedit-toast-message-impression
Expanding and collapsing dialogNone (new feature)postedit-expand and postedit-collapse
Clicking on "view all suggested edits"postedit-link-click with homepage action datapostedit-link-click with suggested-edits action data
Clicking on taskpostedit-task-clickunchanged
Clicking on navigation arrows to navigate through task feedNone (new feature replacing refresh icon)postedit-task-navigation with dir (next, prev) and navigation_type (swipe, click) action dataSimilar to se-task-navigation for suggested edits
Closing dialogpostedit-closeunchanged
Task shownpostedit-impression with newcomerTaskToken action_data and newcomertask eventunchanged

With this task, the post-edit dialog will show the same tasks as before (with the refresh button). Full task feed will come as part of T302335.

Edge cases
  • Structured task flow on desktop currently relies on the post-edit dialog closing to reload the page in order to tear down the StructuredTaskArticleTarget to show the default VE ArticleTarget instead. For structured tasks, the page will now be re-loaded after saving (like mobile).
  • For unstructured tasks, the user can edit the same article again and the post-edit dialog is shown each time the edit is saved. When the user edits again, the drawer should be removed.
  • When there are no more suggestions, the graphic introducing suggested edits should be shown.

Screen Shot 2022-03-14 at 12.46.50 PM.png (520×1 px, 159 KB)

Related Objects

Event Timeline

This goes first to @RHo to design an improved "post-edit dialog" experience.

Did people click the task reload icon? I wonder if the more limited / less obvious browsing options are part of the reason of people not interacting with tasks so much.

In our team meeting today, we decided that we prefer Proposal B because it is consistent with the bottomsheet behavior the user would be used to from structured tasks, and because the user can scroll behind the bottomsheet to see the article without dismissing it. In order to move forward, we need two things from @RHo:

  • We need a design for the desktop version.
  • We need to know how long the banner should display before fading.

Other notes:

RHo raised the priority of this task from High to Needs Triage.Feb 28 2022, 9:22 PM
RHo updated the task description. (Show Details)
RHo renamed this task from Structured tasks: increased flexibility at post-edit dialog to Newcomer tasks: increased flexibility at post-edit dialog.Mar 1 2022, 10:39 AM
RHo updated the task description. (Show Details)

Hi @RHo, can you share the Figma link for the specs (it looks like I can't inspect the prototypes)?

I also wanted to clarify the triggers for collapsing.

  • Should it behave link the image inspector where the only way to collapse it is to click on the header or should it be like the link inspector where clicking anywhere outside also collapses it?
  • My understanding is that this post-edit UI is more like a drawer rather than a dialog. For the current post-edit dialog, hitting ESC key will close the dialog. Since this is not quite a dialog, should it not respond to ESC key? (For structured tasks, we make the inspector behave as if it were part of the editing surface and not a dialog, so ESC will take the user to read mode).
  • Should the drawer be collapsed when the user edits the same article again (for unstructured tasks where currently the post-edit dialog is shown again) ? Or should it be removed entirely?

For the toast message on desktop, the default position for notifications is on the corner. Should we be consistent with other notifications?

Screen Shot 2022-03-03 at 1.14.22 PM.png (984×1 px, 385 KB)

Hi @RHo, can you share the Figma link for the specs (it looks like I can't inspect the prototypes)?

Hi @mewoph - I've added them to the task description but you can also go to the figma page here for Mobile, and here for Desktop.

I also wanted to clarify the triggers for collapsing.

  • Should it behave link the image inspector where the only way to collapse it is to click on the header or should it be like the link inspector where clicking anywhere outside also collapses it?
  • Hi @mewoph, I think it should be like Add image where it's clicking anywhere on the header. And as a side note I think we should also update the Add link inspector so that it only closes on selecting the header only. I think we originally had it auto collapse clicking anywhere outside before because the help was in the header instead of the collapse icon.
  • My understanding is that this post-edit UI is more like a drawer rather than a dialog. For the current post-edit dialog, hitting ESC key will close the dialog. Since this is not quite a dialog, should it not respond to ESC key? (For structured tasks, we make the inspector behave as if it were part of the editing surface and not a dialog, so ESC will take the user to read mode).

Ahh, you bring up a good point, about being able to escape. Initially I was thinking this would be a limitation in that the post-edit would remain collapsed after any suggested edit, but if we are likely to want to show this to newcomers after any edit in the near future it will be extremely irritating and dark pattern not to be able to close it entirely. As such, I want to propose adding an ability to close it via a "close" icon that appears in the collapsed version of this new next edit dialog, which on Desktop would also respond to the ESC key:

Desktop mock
image.png (1×2 px, 871 KB)
Demo
Mobile mock
image.png (1×724 px, 197 KB)
Demo

Would this be feasible as part of the task or shall I make it a separate task for discussion?

  • Should the drawer be collapsed when the user edits the same article again (for unstructured tasks where currently the post-edit dialog is shown again) ? Or should it be removed entirely?

I was thinking we would keep it open, as the user may be ready to try a suggested next edit after their X subsequent edit on an article. Curious if there is a downside to this that I am not seeing though?

For the toast message on desktop, the default position for notifications is on the corner. Should we be consistent with other notifications?

Screen Shot 2022-03-03 at 1.14.22 PM.png (984×1 px, 385 KB)

Yes, I was thinking about this during design. Even though it is slightly less consistent, I would prefer to have this specific success message which refers to the next suggested edit in the centre as it provides a little more of a connection with the next edit suggestion drawer.

Thanks @RHo! We can add the close functionality as a part of this task unless the initial changes can be released without the ability to close.

I was thinking we would keep it open, as the user may be ready to try a suggested next edit after their X subsequent edit on an article. Curious if there is a downside to this that I am not seeing though?

I think one downside is that the drawer might be distracting if the user is typing into the area behind it. I don't think we have this interaction paradigm currently, since for VE, the context item is closed whenever the user interacts with the article surface and for structured tasks, the user can't directly edit the article. Another issue is that when the user edits again and saves while the drawer is open, we might need a different experience for "showing" the post-edit state (since the original animation no longer applies).

Hi @nettrom_WMF, here are the changes that impact instrumentation as well as the corresponding events in the existing flow. Can you confirm whether the proposed events make sense? Thanks!

(table moved to task description)

Hi @mewoph, thanks for putting this all together! These make sense. I was hoping to see some similarities with se-task-navigation and so the referral to recommendedtoolbar_dialog also gets a thumbs up from me!

Let me reply to your question and add a couple of my own:

  • I don't think we need to instrument the postedit-toast-message being dismissed, as we'll be much more interested in user interactions with the suggested edit dialogue than the toast itself.
  • Will swiping be possible on the mobile version (perhaps mostly a question for @RHo)? If so, then having se-task-navigation-swipe like we do for Suggested Edits would make sense.
  • For "Task shown", bottom row in the table, would the event be postedit-task-impression with newcomerTaskToken in action_data, and a corresponding event in the newcomertask stream, similarly as we currently have? I'd just like to doublecheck to make sure :)

Thanks for confirming @nettrom_WMF! For swiping, I just realized that we have two paradigms for tracking: separate actions (for suggested edits) and as an additional navigation_type action_data with either swipe or click (onboarding dialog and link inspector). My recollection is that the latter paradigm was added for events where we weren't sure whether we need to analyze the swipe navigation specifically, but the data is there just in case. Would it make sense to add the latter here so that we have some flexibility?

For "Task shown", bottom row in the table, would the event be postedit-task-impression with newcomerTaskToken in action_data, and a corresponding event in the newcomertask stream, similarly as we currently have? I'd just like to doublecheck to make sure :)

Yes, I forgot to add the corresponding impression event. I'll update the table accordingly.

For swiping, I just realized that we have two paradigms for tracking: separate actions (for suggested edits) and as an additional navigation_type action_data with either swipe or click (onboarding dialog and link inspector). My recollection is that the latter paradigm was added for events where we weren't sure whether we need to analyze the swipe navigation specifically, but the data is there just in case. Would it make sense to add the latter here so that we have some flexibility?

Ah, thanks for being aware of the various paradigms we have used! I think it makes sense to go with the latter here as well, as I expect we'll be focused on the action of navigating to a different tasks and not whether they tapped/clicked/swiped to do so. In case we later want to study users swiping, we have the information there to distinguish it, as you mention. Thanks again for remembering this so we could figure it out!

Thanks @RHo! We can add the close functionality as a part of this task unless the initial changes can be released without the ability to close.

Hiya - per conversation with @MMiller_WMF, I'm going to move the close functionality into this new task: T303416: Newcomer Tasks: Enable closing collapsed Post-edit dialog

I was thinking we would keep it open, as the user may be ready to try a suggested next edit after their X subsequent edit on an article. Curious if there is a downside to this that I am not seeing though?

I think one downside is that the drawer might be distracting if the user is typing into the area behind it. I don't think we have this interaction paradigm currently, since for VE, the context item is closed whenever the user interacts with the article surface and for structured tasks, the user can't directly edit the article. Another issue is that when the user edits again and saves while the drawer is open, we might need a different experience for "showing" the post-edit state (since the original animation no longer applies).

Hmm I think I am not understanding the scenario described. At the moment, when I edit the same article again in an unstructured task, the post-edit dialog is shown again, so the change would be that instead of a dialog it is a drawer, but in both cases the post-edit element appears when the user is in Read-mode, so I don't get how the user would be editing behind the drawer? Once they select "edit" to edit again, the drawer would disappear. Is this something that should be clarified on the task?

Hi @RHo — maybe I misunderstood.

Should the drawer be collapsed when the user edits the same article again (for unstructured tasks where currently the post-edit dialog is shown again) ? Or should it be removed entirely?
I was thinking we would keep it open, as the user may be ready to try a suggested next edit after their X subsequent edit on an article. Curious if there is a downside to this that I am not seeing though?

My question was whether the drawer should be collapsed or removed when the user edits again. I thought your answer was that it should remain open. To confirm, the drawer should be removed entirely (not just collapsed) when the user edits again?

Hi @RHo — maybe I misunderstood.

Should the drawer be collapsed when the user edits the same article again (for unstructured tasks where currently the post-edit dialog is shown again) ? Or should it be removed entirely?
I was thinking we would keep it open, as the user may be ready to try a suggested next edit after their X subsequent edit on an article. Curious if there is a downside to this that I am not seeing though?

My question was whether the drawer should be collapsed or removed when the user edits again. I thought your answer was that it should remain open. To confirm, the drawer should be removed entirely (not just collapsed) when the user edits again?

Ah yes, sorry I should have been clearer in the task description but that is correct. The drawer should be removed entirely (just like the post-edit dialog) if the user navigates away from Read, such as by going to Edit again.

Thanks for confirming @RHo, I updated the task description to account for this case.

mewoph updated the task description. (Show Details)

Hi @RHo — I just realized there is another edge case for mobile now that the post-edit drawer is always visible. For structured tasks, after the edit is saved, the help panel button is still visible.

This is somewhat of an existing problem for narrower mobile screens (where the drawer takes up the full width) since the help panel button is shown along with the drawer but once the drawer goes away, the button is visible).

Screen Shot 2022-03-09 at 2.24.32 PM.png (1×1 px, 810 KB)

How should we handle this case?

postedit_mobile.gif (1×754 px, 1 MB)

One simple solution would be to let the drawer cover the help button make the help button visible upon collapsing. This would help us not deal with completing overlays (help panel vs post-edit drawer) if the help panel can only opened upon collapsing the post-edit drawer.

Default help button positionUpdated help button position
Screen Shot 2022-03-09 at 2.33.12 PM.png (1×758 px, 186 KB)
Screen Shot 2022-03-09 at 2.32.58 PM.png (1×754 px, 188 KB)

Hi @nettrom_WMF, I just realized that the post-edit flow uses the legacy schema so it doesn't have active_interface field. I added the table to the task description. The two proposed changes are:

  • postedit-toast-message-impression instead of impression with postedit-toast-message active_interface
  • postedit-expand and postedit-collapse instead of expand and collapse with postedit_dialog active_interface

This would make this not consistent with recommendedimagetoolbar_dialog.

Alternatively, would it make sense for us to add active_interface in action_data and use impression, expand and collapse like the link/image suggestion schemas?

Hi @RHo — I just realized there is another edge case for mobile now that the post-edit drawer is always visible. For structured tasks, after the edit is saved, the help panel button is still visible.
This is somewhat of an existing problem for narrower mobile screens (where the drawer takes up the full width) since the help panel button is shown along with the drawer but once the drawer goes away, the button is visible).

Screen Shot 2022-03-09 at 2.24.32 PM.png (1×1 px, 810 KB)

How should we handle this case?
postedit_mobile.gif (1×754 px, 1 MB)

One simple solution would be to let the drawer cover the help button make the help button visible upon collapsing. This would help us not deal with completing overlays (help panel vs post-edit drawer) if the help panel can only opened upon collapsing the post-edit drawer.

Default help button positionUpdated help button position
Screen Shot 2022-03-09 at 2.33.12 PM.png (1×758 px, 186 KB)
Screen Shot 2022-03-09 at 2.32.58 PM.png (1×754 px, 188 KB)

Hi @mewoph - I think you may have found a bug that I didn't realise existed in that I don't believe the help button should be shown at all (Desktop or Mobile) after completing the structured task. Also, as I was trying to see whether it would make sense to have the help button contents show up in read mode after the suggested edit was made, I noticed that selecting the Help button doesn't actually work because it disappears as soon as it is clicked (testwiki):

Would filing a task to simply not show the help button after an edit is made solve this? Otherwise, I agree with your proposal to make it visible when the button is collapsed on mobile.

Hi @RHo, I filed T303553: Help button should not show up when the user edits the suggested article again so we can fix the existing issue before rolling out this change which makes the issue more pronounced.

Change 769769 had a related patch set uploaded (by MewOphaswongse; author: MewOphaswongse):

[schemas/event/secondary@master] Help panel: update schema to reflect new post-edit experience

https://gerrit.wikimedia.org/r/769769

Change 769792 had a related patch set uploaded (by MewOphaswongse; author: MewOphaswongse):

[mediawiki/extensions/GrowthExperiments@master] [DNM] Post-edit: show PostEditPanel inside a collapsible drawer for both desktop and mobile

https://gerrit.wikimedia.org/r/769792

Hi @RHo, should we show an alternate UI when there are no suggestions to show? The existing dialog simply hides the task card and the new implementation can inherit this behavior. However it's a bit weird to have the expand/collapse functionality when there's nothing to collapse and the drawer might be too short for it to be noticeable.

Hi @mewoph - I was thinking eventually we may want to enable users to change filters to ensure they can bring up more suggested edits directly, but in the interim we could provide a graphic that (re)introduces the suggested edits feature:

image.png (392×1 px, 88 KB)

How does that sound?

Hi @RHo, for this particular task, the interim solution sounds good. I'll update the task description.

For the copy, would it be possible to re-word it so that $Sitename is used as a noun rather than an adjective so that it's easier to translate? Perhaps instead of "$sitename users", we can have "users of $sitename"?

Task typeDesktop w/editsDesktop w/o editsMobile w/editsMobile w/o edits
Copyedit
postedit_desktop_copyedit.png (1×1 px, 531 KB)
N/A
postedit_mobile_copyedit.png (1×748 px, 202 KB)
N/A
Add link
postedit_desktop_addlink_accepted.png (1×2 px, 793 KB)
postedit_desktop_addlink_rejected.png (1×2 px, 878 KB)
postedit_mobile_addlink_accepted.png (1×745 px, 194 KB)
postedit_mobile_addlink_rejected.png (1×752 px, 192 KB)
Add image
postedit_desktop_addimage_accepted.png (1×2 px, 1 MB)
postedit_desktop_addImage_rejected.png (1×2 px, 534 KB)
postedit_mobile_addlink_accepted.png (1×745 px, 194 KB)
postedit_mobile_addlink_rejected.png (1×752 px, 192 KB)

Hi @nettrom_WMF, I just realized that the post-edit flow uses the legacy schema so it doesn't have active_interface field. I added the table to the task description. The two proposed changes are:

  • postedit-toast-message-impression instead of impression with postedit-toast-message active_interface
  • postedit-expand and postedit-collapse instead of expand and collapse with postedit_dialog active_interface

This would make this not consistent with recommendedimagetoolbar_dialog.

Alternatively, would it make sense for us to add active_interface in action_data and use impression, expand and collapse like the link/image suggestion schemas?

I think using the postedit-* naming of the actions makes good sense. It fits the current paradigm of the schema and is easy to work with. I do like the setup of the structured task schemas with active_interface, action, and action_data fields, but I think that's probably best left for a future change to the HelpPanel's instrumentation, such as moving away from the legacy schema setup.

So in summary, I'm happy with moving forward with what's currently proposed in the task description! :)

Hi @RHo, for this particular task, the interim solution sounds good. I'll update the task description.

For the copy, would it be possible to re-word it so that $Sitename is used as a noun rather than an adjective so that it's easier to translate? Perhaps instead of "$sitename users", we can have "users of $sitename"?

Thanks Mew, and that revised copy sounds good to me. I created a task for the addition of filters on the dialog on T303824: Newcomer tasks: Enable filter changes directly on the post-edit dialog when there no suggested edits, but it is not a priority at the moment

Change 769769 merged by jenkins-bot:

[schemas/event/secondary@master] Help panel: update schema to reflect new post-edit experience

https://gerrit.wikimedia.org/r/769769

@nettrom_WMF @MMiller_WMF @RHo is there any concern with this experience affecting data collection for ongoing experiments (add image, add link, etc)? If not, the patches can be merged either for this week's train or next week's.

@kostajh -- thank you for checking, but there is not a concern around that. The change will happen to the treatment and control groups at the same time, so it won't affect the experiment. We'll just want to know exactly what day the change is in place so we can see whether the funnel analysis changes after.

Change 769792 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Post-edit: show PostEditPanel inside a collapsible drawer for both desktop and mobile

https://gerrit.wikimedia.org/r/769792

@RHo - a minor point: currently, the post-edit dialog says Edit another suggestion: Your mockups refer to Edit another article. Going through the comments above, I assume it's ok to have that difference between mockups and the implementation, especially since the current implementation matches production.

current production wmf.10mockupsbetalabs
Screen Shot 2022-05-06 at 3.17.00 PM.png (830×1 px, 124 KB)
Screen Shot 2022-05-06 at 4.53.58 PM.png (1×1 px, 708 KB)
Screen Shot 2022-05-06 at 3.32.17 PM.png (738×1 px, 93 KB)

(click on the animated gif below) This is an illustration of timing of different elements.

post_edit_success_msg2.gif (771×1 px, 559 KB)

Quality gates - the screenshots are from cswiki beta

  • Mobile and Desktop Post-edit UI on the Add image task when the quality gate quote is reached (with and without an edit)

Image quality gate reached|Add link quality gate reached

Screen Shot 2022-05-06 at 5.52.54 PM.png (1×1 px, 564 KB)
Mobile:
Screenshot_20220509-175454_Chrome.jpg (2×1 px, 375 KB)
Screen Shot 2022-05-09 at 5.13.15 PM.png (1×1 px, 370 KB)
Mobile:
Screenshot_20220509-175454_Chrome.jpg (2×1 px, 375 KB)

Thanks for checking @Etonkovidova, agree we can stick with the prod copy.
And looks good testing on enwiki beta today on Desktop and Mobile (simulator) with the additional close action (split into T303416) added too:

Desktop Mobile

@RHo - a minor point: currently, the post-edit dialog says Edit another suggestion: Your mockups refer to Edit another article. Going through the comments above, I assume it's ok to have that difference between mockups and the implementation, especially since the current implementation matches production.

current production wmf.10mockupsbetalabs
Screen Shot 2022-05-06 at 3.17.00 PM.png (830×1 px, 124 KB)
Screen Shot 2022-05-06 at 4.53.58 PM.png (1×1 px, 708 KB)
Screen Shot 2022-05-06 at 3.32.17 PM.png (738×1 px, 93 KB)

(click on the animated gif below) This is an illustration of timing of different elements.

post_edit_success_msg2.gif (771×1 px, 559 KB)