Page MenuHomePhabricator
Create Task
Maniphest T229079

Instrument each step of mobile VE edit flow
Closed, ResolvedPublic
Actions

Description

We intend for the improvements we are making to the mobile VE editing toolbar and the steps within the edit flow [1] to help a greater number of people complete their edits.

In order for us to be able to evaluate how effective our efforts have been, we'll need to be able to answer: How do people "move through" the edit flow?

And for us to be able to answer this question, we'll need instrumentation to track the actual actions/steps contributors are taking within the "edit flow".

What we know

Currently, we know:

  • When the editing interface starts to load: init
  • When the editing interface is ready to be interacted with: ready
  • When a contributor advances to the "Save your changes" screen/dialog: saveIntent
  • When a contributor tries to save an edit: saveAttempt
  • When a contributor successfully saves an edit: saveSuccess
  • When a contributor unsuccessfully ties to save an edit: saveFailure
  • When a contributor aborts their edit: abort
    • The conditions under which a contributor aborts. Examples:
      • No changes have been made: nochange
      • Editor has be initialised but is not yet ready: preinit
      • User is switching from one editor to another, retaining changes: switchwith
      • User is switching from one editor to another, discarding changes: switchwithout
      • User is switching from one editor to another, and the content block is unchanged / transaction stack is empty: switchnochange
      • User is exiting the editor, discarding changes: abandon
      • User is exiting the editor, discarding changes, after a saveAttempt has been initiated but before saveSuccess/saveFailure.": abandonMidsave

Source: Schema:EditAttemptStep

We would like to know ("Done")

After this work is complete, we would like to know:

  • When a contributor "starts" an edit
    • "Starts" could mean when a contributor makes any kind of change to the article
    • Read: when the first transaction is made
  • When a contributors taps “<” in “Save your changes” dialog
  • When a contributors taps “<” in “Review your changes” view
  • When a contributor taps to "Preview"/"Review" their changes
  • When a contributor taps "Visual" or "Wikitext" in “Review your changes” view
  • When a contributor taps the "This is a minor edit" checkbox in the Save dialog
  • When a contributor taps the "Watch this page" checkbox in the Save dialog
  • When a contributor taps the “>” in v1 of the toolbar

This work also relates to our investigation into mobile VE's loading performance: T227930.

  1. T211255: [EPIC] Mobile VE toolbar improvements

Details

ProjectBranchLines +/-Subject
mediawiki/extensions/VisualEditormaster+32 -2MWSaveDialog: add more instrumentation
mediawiki/extensions/WikiEditormaster+18 -5Implement firstChange logging
mediawiki/extensions/MobileFrontendmaster+18 -4Add new EditAttemptStep stage: firstChange
mediawiki/extensions/VisualEditormaster+15 -0Add new EditAttemptStep stage: firstChange
mediawiki/extensions/WikimediaEventsmaster+1 -1Update EditAttemptStep version
mediawiki/extensions/WikimediaEventsmaster+1 -1Update EditAttemptStep
Customize query in gerrit

Related Objects
Search...

Event Timeline

ppelberg created this task.Jul 26 2019, 12:45 AM
Restricted Application added a project: VisualEditor. · View Herald TranscriptJul 26 2019, 12:45 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
ppelberg renamed this task from Instrument each step of the edit flow to Instrument each step of mobile VE edit flow.Jul 26 2019, 12:49 AM
ppelberg updated the task description. (Show Details)Jul 26 2019, 6:07 PM
ppelberg updated the task description. (Show Details)Jul 26 2019, 6:23 PM
ppelberg updated the task description. (Show Details)Jul 30 2019, 7:52 PM
ppelberg mentioned this in T221316: Create measurement specifications and experiment plan.Jul 31 2019, 2:06 PM
ppelberg updated the task description. (Show Details)Aug 1 2019, 11:51 PM
JTannerWMF edited projects, added VisualEditor (Current work); removed VisualEditor.Aug 8 2019, 4:40 PM
JTannerWMF added subscribers: DLynch, Esanders, JTannerWMF.

Moving this to our current workboard for @DLynch or @Esanders to pick up

JTannerWMF moved this task from Incoming to Ready for Development on the VisualEditor (Current work) board.Aug 21 2019, 11:54 AM
ppelberg added a comment.EditedAug 29 2019, 3:11 PM

Updates to task description to include the following events in the ...we would like to know section:

  • Tapping “<” in “Save your changes” dialog
  • Tapping “<” in “Review your changes” view
  • Tapping “Visual” or "Wikitext" in "Review your changes" view
  • Tapping “>” in the v1 toolbar
ppelberg updated the task description. (Show Details)Aug 29 2019, 3:17 PM
ppelberg updated the task description. (Show Details)Aug 29 2019, 6:17 PM
ppelberg updated the task description. (Show Details)Sep 4 2019, 6:54 PM
ppelberg added subscribers: nshahquinn-wmf, MNeisler.

Updates to task description:

  • Removing T229030 as a dependency
  • Removing "⚠️This is a tracker task... message
  • Removing "This is likely to change considering we are planning to show the visual diff screen view by default. See the description of: T229030"

Documenting notes/decisions from a conversation today with @MNeisler and @Neil_P._Quinn_WMF:

  • The following events should should live in Schema:VisualEditorFeaureUse:
    • "When a contributor "starts" an edit"
  • The following events should should live in Schema:EditAttemptStep
    • When a contributors taps “<” in “Save your changes” dialog
    • When a contributors taps “<” in “Review your changes” view
    • When a contributor taps to "Preview"/"Review" their changes
    • When a contributor taps "Visual" or "Wikitext" in “Review your changes” view
    • When a contributor taps the "This is a minor edit" checkbox in the Save dialog
    • When a contributor taps the "Watch this page" checkbox in the Save dialog
    • When a contributor taps the “>” in v1 of the toolbar

@DLynch, are there any other questions you are needing answers to before work on adding instrumentation for the events listed in the NAME above can start?

nshahquinn-wmf added a comment.Sep 5 2019, 12:41 PM
In T229079#5466100, @ppelberg wrote:

Documenting notes/decisions from a conversation today with @MNeisler and @Neil_P._Quinn_WMF:

  • The following events should should live in Schema:VisualEditorFeaureUse:
    • "When a contributor "starts" an edit"
  • The following events should should live in Schema:EditAttemptStep
    • When a contributors taps “<” in “Save your changes” dialog
    • When a contributors taps “<” in “Review your changes” view
    • When a contributor taps to "Preview"/"Review" their changes
    • When a contributor taps "Visual" or "Wikitext" in “Review your changes” view
    • When a contributor taps the "This is a minor edit" checkbox in the Save dialog
    • When a contributor taps the "Watch this page" checkbox in the Save dialog
    • When a contributor taps the “>” in v1 of the toolbar

That's actually backwards; we thought starting an edit should be in EditAttemptStep and the rest should be in VisualEditorFeatureUse 😁

ppelberg added a comment.EditedSep 5 2019, 4:10 PM
In T229079#5467820, @Neil_P._Quinn_WMF wrote:

That's actually backwards; we thought starting an edit should be in EditAttemptStep and the rest should be in VisualEditorFeatureUse 😁

Shoot, yes. Good catch, @Neil_P._Quinn_WMF – thank you. The events should be logged as follows...

  • The following events should should live in Schema:VisualEditorFeaureUse:
    • When a contributors taps “<” in “Save your changes” dialog
    • When a contributors taps “<” in “Review your changes” view
    • When a contributor taps to "Preview"/"Review" their changes
    • When a contributor taps "Visual" or "Wikitext" in “Review your changes” view
    • When a contributor taps the "This is a minor edit" checkbox in the Save dialog
    • When a contributor taps the "Watch this page" checkbox in the Save dialog
    • When a contributor taps the “>” in v1 of the toolbar
nshahquinn-wmf added a project: Product-Analytics.Sep 6 2019, 4:30 PM
nshahquinn-wmf moved this task from Triage to Tracking on the Product-Analytics board.
DLynch added a comment.Sep 9 2019, 2:29 PM

@Neil_P._Quinn_WMF / @MNeisler : Given that the "starts an edit" part requires a change to the EditAttemptStep schema, could you verify that all I'd need to do is add, say, "firstTransaction" to the action enum, and add a first_transaction_timing integer? No other extra data required?

gerritbot added a comment.Sep 9 2019, 3:53 PM

Change 535218 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/VisualEditor@master] MWSaveDialog: add more instrumentation

https://gerrit.wikimedia.org/r/535218

gerritbot added a project: Patch-For-Review.Sep 9 2019, 3:53 PM
DLynch added a comment.Sep 9 2019, 3:53 PM

When a contributors taps...

“<” in “Save your changes” dialogadded in patch: mwSave.dialog-abort
“<” in “Review your changes” viewadded in patch: mwSave.dialog-review
to "Preview"/"Review" their changesadded in patch: mwSave.dialog-review/preview
"Visual" or "Wikitext" in “Review your changes” viewadded in patch: mwSave.review-switch-source / mwSave.review-switch-visual
the "This is a minor edit" checkbox in the Save dialogadded in patch: mwSave.checkbox-wpMinoredit
the "Watch this page" checkbox in the Save dialogadded in patch: mwSave.checkbox-wpWatchthis
the “>” in v1 of the toolbaralready present in EditAttemptStep as saveIntent

@ppelberg do you actually want that last one duplicated?

DLynch moved this task from Ready for Development to Code review on the VisualEditor (Current work) board.Sep 9 2019, 3:53 PM
ppelberg added a comment.Sep 9 2019, 9:10 PM
In T229079#5475532, @DLynch wrote:

When a contributors taps...

“<” in “Save your changes” dialogadded in patch: mwSave.dialog-abort
“<” in “Review your changes” viewadded in patch: mwSave.dialog-review
to "Preview"/"Review" their changesadded in patch: mwSave.dialog-review/preview
"Visual" or "Wikitext" in “Review your changes” viewadded in patch: mwSave.review-switch-source / mwSave.review-switch-visual
the "This is a minor edit" checkbox in the Save dialogadded in patch: mwSave.checkbox-wpMinoredit
the "Watch this page" checkbox in the Save dialogadded in patch: mwSave.checkbox-wpWatchthis
the “>” in v1 of the toolbaralready present in EditAttemptStep as saveIntent

@ppelberg do you actually want that last one duplicated?

@DLynch, I hadn't realized saveIntent mapped to a contributing tapping the “>” button in the toolbar (see below). In which case, "no," I see no reason to create a new/another event for one that already exists. I appreciate you checking.

MNeisler added a comment.Sep 9 2019, 10:18 PM
In T229079#5475160, @DLynch wrote:

@Neil_P._Quinn_WMF / @MNeisler : Given that the "starts an edit" part requires a change to the EditAttemptStep schema, could you verify that all I'd need to do is add, say, "firstTransaction" to the action enum, and add a first_transaction_timing integer? No other extra data required?

@DLynch - Yes, that seems right. No extra data should be required.

DLynch added a comment.Sep 10 2019, 12:04 AM

@MNeisler - Sorry, I didn't think to ask -- for the timing, would it be more helpful if I stored time-since-initialization or time-since-ready? That'd sort of be whether this event is conceptually grouped with the editor-loading cluster or the saving cluster.

DLynch added a comment.Sep 10 2019, 12:07 AM

I have updated the schema for now, with time-since-ready: https://meta.wikimedia.org/w/index.php?title=Schema:EditAttemptStep&diff=19364181&oldid=19273145

The answer to that question will just require a documentation change if I picked incorrectly.

gerritbot added a comment.Sep 10 2019, 12:32 AM

Change 535319 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/WikimediaEvents@master] Update EditAttemptStep

https://gerrit.wikimedia.org/r/535319

gerritbot added a comment.Sep 10 2019, 12:33 AM

Change 535320 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/VisualEditor@master] Add new EditAttemptStep stage: firstTransaction

https://gerrit.wikimedia.org/r/535320

gerritbot added a comment.Sep 10 2019, 12:34 AM

Change 535321 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/MobileFrontend@master] Add new EditAttemptStep stage: firstTransaction

https://gerrit.wikimedia.org/r/535321

DLynch added a comment.Sep 10 2019, 12:39 AM

Okay! That cluster of patches implements the firstTransaction step on desktop and mobile. It logs it with the timing as "time since the document was ready".

gerritbot added a comment.Sep 10 2019, 12:48 AM

Change 535319 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Update EditAttemptStep

https://gerrit.wikimedia.org/r/535319

ReleaseTaggerBot added a project: MW-1.34-notes (1.34.0-wmf.22; 2019-09-10).Sep 10 2019, 1:00 AM
Esanders added a comment.Sep 10 2019, 1:40 PM

I assume this should be tracked in non-VE editors as well. In which case "firstTransaction" might be renamed "firstEdit" or "firstChange" as non VE surfaces don't necessarily use transactions. We should also write patches for those editors.

DLynch added a comment.Sep 10 2019, 2:01 PM

We should also write patches for those editors.

I did have that as a to-do, just lower priority than the initial round of VE testing.

gerritbot added a comment.Sep 10 2019, 2:06 PM

Change 535599 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/WikimediaEvents@master] Update EditAttemptStep version

https://gerrit.wikimedia.org/r/535599

gerritbot added a comment.Sep 10 2019, 2:26 PM

Change 535599 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Update EditAttemptStep version

https://gerrit.wikimedia.org/r/535599

gerritbot added a comment.Sep 10 2019, 2:35 PM

Change 535612 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/WikiEditor@master] Implement firstChange logging

https://gerrit.wikimedia.org/r/535612

DLynch added a comment.Sep 10 2019, 2:52 PM

Okay, current patches implement firstChange in: desktop VisualEditor, desktop WikiEditor, MobileFrontend wikitext and visual.

ReleaseTaggerBot edited projects, added MW-1.34-notes (1.34.0-wmf.23; 2019-09-17); removed MW-1.34-notes (1.34.0-wmf.22; 2019-09-10).Sep 10 2019, 3:00 PM
gerritbot added a comment.Sep 11 2019, 9:14 PM

Change 535321 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Add new EditAttemptStep stage: firstChange

https://gerrit.wikimedia.org/r/535321

gerritbot added a comment.Sep 12 2019, 3:19 PM

Change 535320 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Add new EditAttemptStep stage: firstChange

https://gerrit.wikimedia.org/r/535320

gerritbot added a comment.Sep 12 2019, 6:08 PM

Change 535612 merged by jenkins-bot:
[mediawiki/extensions/WikiEditor@master] Implement firstChange logging

https://gerrit.wikimedia.org/r/535612

ppelberg added a subtask: T232790: Verify editing flow events are firing properly.Sep 12 2019, 10:23 PM
ppelberg merged a task: T221321: Instrument "refreshed" toolbar.Sep 13 2019, 11:57 PM
gerritbot added a comment.Sep 18 2019, 3:13 PM

Change 535218 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] MWSaveDialog: add more instrumentation

https://gerrit.wikimedia.org/r/535218

ReleaseTaggerBot edited projects, added MW-1.34-notes (1.34.0-wmf.24; 2019-09-24); removed MW-1.34-notes (1.34.0-wmf.23; 2019-09-17).Sep 18 2019, 4:00 PM
DLynch moved this task from Code review to Product owner review on the VisualEditor (Current work) board.Sep 18 2019, 4:02 PM
DLynch edited projects, added Skipped QA; removed Patch-For-Review.

T232790 is basically the QA process, after all.

ppelberg mentioned this in T232790: Verify editing flow events are firing properly.Sep 19 2019, 9:23 PM
ppelberg mentioned this in T233823: Mobile VE edit flow: instrumentation post-deployment data checks.Sep 25 2019, 3:25 PM
dchan moved this task from Current work to Analysis on the VisualEditor board.Oct 6 2019, 6:09 AM
dchan edited projects, added VisualEditor; removed VisualEditor (Current work).
JTannerWMF edited projects, added VisualEditor (Current work); removed VisualEditor.Oct 10 2019, 3:50 PM

It looks like this is ready for Peter's review

ppelberg closed this task as Resolved.Oct 23 2019, 1:10 AM
ppelberg closed subtask T232790: Verify editing flow events are firing properly as Resolved.Feb 15 2020, 1:59 AM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL