Page MenuHomePhabricator

Instrument each step of mobile VE edit flow
Closed, ResolvedPublic

Description

We intend for the improvements we are making to the mobile VE editing toolbar and the steps within the edit flow [1] to help a greater number of people complete their edits.

In order for us to be able to evaluate how effective our efforts have been, we'll need to be able to answer: How do people "move through" the edit flow?

And for us to be able to answer this question, we'll need instrumentation to track the actual actions/steps contributors are taking within the "edit flow".

What we know

Currently, we know:

  • When the editing interface starts to load: init
  • When the editing interface is ready to be interacted with: ready
  • When a contributor advances to the "Save your changes" screen/dialog: saveIntent
  • When a contributor tries to save an edit: saveAttempt
  • When a contributor successfully saves an edit: saveSuccess
  • When a contributor unsuccessfully ties to save an edit: saveFailure
  • When a contributor aborts their edit: abort
    • The conditions under which a contributor aborts. Examples:
      • No changes have been made: nochange
      • Editor has be initialised but is not yet ready: preinit
      • User is switching from one editor to another, retaining changes: switchwith
      • User is switching from one editor to another, discarding changes: switchwithout
      • User is switching from one editor to another, and the content block is unchanged / transaction stack is empty: switchnochange
      • User is exiting the editor, discarding changes: abandon
      • User is exiting the editor, discarding changes, after a saveAttempt has been initiated but before saveSuccess/saveFailure.": abandonMidsave

Source: Schema:EditAttemptStep

We would like to know ("Done")

After this work is complete, we would like to know:

  • When a contributor "starts" an edit
    • "Starts" could mean when a contributor makes any kind of change to the article
    • Read: when the first transaction is made
  • When a contributors taps “<” in “Save your changes” dialog
  • When a contributors taps “<” in “Review your changes” view
  • When a contributor taps to "Preview"/"Review" their changes
  • When a contributor taps "Visual" or "Wikitext" in “Review your changes” view
  • When a contributor taps the "This is a minor edit" checkbox in the Save dialog
  • When a contributor taps the "Watch this page" checkbox in the Save dialog
  • When a contributor taps the “>” in v1 of the toolbar

This work also relates to our investigation into mobile VE's loading performance: T227930.


  1. T211255: [EPIC] Mobile VE toolbar improvements

Details

Related Gerrit Patches:
mediawiki/extensions/VisualEditor : masterMWSaveDialog: add more instrumentation
mediawiki/extensions/WikiEditor : masterImplement firstChange logging
mediawiki/extensions/MobileFrontend : masterAdd new EditAttemptStep stage: firstChange
mediawiki/extensions/VisualEditor : masterAdd new EditAttemptStep stage: firstChange
mediawiki/extensions/WikimediaEvents : masterUpdate EditAttemptStep version
mediawiki/extensions/WikimediaEvents : masterUpdate EditAttemptStep

Event Timeline

Restricted Application added a project: VisualEditor. · View Herald TranscriptJul 26 2019, 12:45 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
ppelberg renamed this task from Instrument each step of the edit flow to Instrument each step of mobile VE edit flow.Jul 26 2019, 12:49 AM
ppelberg updated the task description. (Show Details)Jul 26 2019, 6:07 PM
ppelberg updated the task description. (Show Details)Jul 26 2019, 6:23 PM
ppelberg updated the task description. (Show Details)Jul 30 2019, 7:52 PM
ppelberg updated the task description. (Show Details)Aug 1 2019, 11:51 PM
JTannerWMF added subscribers: DLynch, Esanders, JTannerWMF.

Moving this to our current workboard for @DLynch or @Esanders to pick up

ppelberg added a comment.EditedAug 29 2019, 3:11 PM

Updates to task description to include the following events in the ...we would like to know section:

  • Tapping “<” in “Save your changes” dialog
  • Tapping “<” in “Review your changes” view
  • Tapping “Visual” or "Wikitext" in "Review your changes" view
  • Tapping “>” in the v1 toolbar
ppelberg updated the task description. (Show Details)Aug 29 2019, 3:17 PM
ppelberg updated the task description. (Show Details)Aug 29 2019, 6:17 PM
ppelberg updated the task description. (Show Details)Sep 4 2019, 6:54 PM
ppelberg added subscribers: Neil_P._Quinn_WMF, MNeisler.

Updates to task description:

  • Removing T229030 as a dependency
  • Removing "⚠️This is a tracker task... message
  • Removing "This is likely to change considering we are planning to show the visual diff screen view by default. See the description of: T229030"

Documenting notes/decisions from a conversation today with @MNeisler and @Neil_P._Quinn_WMF:

  • The following events should should live in Schema:VisualEditorFeaureUse:
    • "When a contributor "starts" an edit"
  • The following events should should live in Schema:EditAttemptStep
    • When a contributors taps “<” in “Save your changes” dialog
    • When a contributors taps “<” in “Review your changes” view
    • When a contributor taps to "Preview"/"Review" their changes
    • When a contributor taps "Visual" or "Wikitext" in “Review your changes” view
    • When a contributor taps the "This is a minor edit" checkbox in the Save dialog
    • When a contributor taps the "Watch this page" checkbox in the Save dialog
    • When a contributor taps the “>” in v1 of the toolbar

@DLynch, are there any other questions you are needing answers to before work on adding instrumentation for the events listed in the NAME above can start?

Documenting notes/decisions from a conversation today with @MNeisler and @Neil_P._Quinn_WMF:

  • The following events should should live in Schema:VisualEditorFeaureUse:
    • "When a contributor "starts" an edit"
  • The following events should should live in Schema:EditAttemptStep
    • When a contributors taps “<” in “Save your changes” dialog
    • When a contributors taps “<” in “Review your changes” view
    • When a contributor taps to "Preview"/"Review" their changes
    • When a contributor taps "Visual" or "Wikitext" in “Review your changes” view
    • When a contributor taps the "This is a minor edit" checkbox in the Save dialog
    • When a contributor taps the "Watch this page" checkbox in the Save dialog
    • When a contributor taps the “>” in v1 of the toolbar

That's actually backwards; we thought starting an edit should be in EditAttemptStep and the rest should be in VisualEditorFeatureUse 😁

ppelberg added a comment.EditedSep 5 2019, 4:10 PM

That's actually backwards; we thought starting an edit should be in EditAttemptStep and the rest should be in VisualEditorFeatureUse 😁

Shoot, yes. Good catch, @Neil_P._Quinn_WMF – thank you. The events should be logged as follows...

  • The following events should should live in Schema:VisualEditorFeaureUse:
    • When a contributors taps “<” in “Save your changes” dialog
    • When a contributors taps “<” in “Review your changes” view
    • When a contributor taps to "Preview"/"Review" their changes
    • When a contributor taps "Visual" or "Wikitext" in “Review your changes” view
    • When a contributor taps the "This is a minor edit" checkbox in the Save dialog
    • When a contributor taps the "Watch this page" checkbox in the Save dialog
    • When a contributor taps the “>” in v1 of the toolbar
DLynch added a comment.Sep 9 2019, 2:29 PM

@Neil_P._Quinn_WMF / @MNeisler : Given that the "starts an edit" part requires a change to the EditAttemptStep schema, could you verify that all I'd need to do is add, say, "firstTransaction" to the action enum, and add a first_transaction_timing integer? No other extra data required?

Change 535218 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/VisualEditor@master] MWSaveDialog: add more instrumentation

https://gerrit.wikimedia.org/r/535218

DLynch added a comment.Sep 9 2019, 3:53 PM

When a contributors taps...

“<” in “Save your changes” dialogadded in patch: mwSave.dialog-abort
“<” in “Review your changes” viewadded in patch: mwSave.dialog-review
to "Preview"/"Review" their changesadded in patch: mwSave.dialog-review/preview
"Visual" or "Wikitext" in “Review your changes” viewadded in patch: mwSave.review-switch-source / mwSave.review-switch-visual
the "This is a minor edit" checkbox in the Save dialogadded in patch: mwSave.checkbox-wpMinoredit
the "Watch this page" checkbox in the Save dialogadded in patch: mwSave.checkbox-wpWatchthis
the “>” in v1 of the toolbaralready present in EditAttemptStep as saveIntent

@ppelberg do you actually want that last one duplicated?

When a contributors taps...

“<” in “Save your changes” dialogadded in patch: mwSave.dialog-abort
“<” in “Review your changes” viewadded in patch: mwSave.dialog-review
to "Preview"/"Review" their changesadded in patch: mwSave.dialog-review/preview
"Visual" or "Wikitext" in “Review your changes” viewadded in patch: mwSave.review-switch-source / mwSave.review-switch-visual
the "This is a minor edit" checkbox in the Save dialogadded in patch: mwSave.checkbox-wpMinoredit
the "Watch this page" checkbox in the Save dialogadded in patch: mwSave.checkbox-wpWatchthis
the “>” in v1 of the toolbaralready present in EditAttemptStep as saveIntent

@ppelberg do you actually want that last one duplicated?

@DLynch, I hadn't realized saveIntent mapped to a contributing tapping the “>” button in the toolbar (see below). In which case, "no," I see no reason to create a new/another event for one that already exists. I appreciate you checking.

@Neil_P._Quinn_WMF / @MNeisler : Given that the "starts an edit" part requires a change to the EditAttemptStep schema, could you verify that all I'd need to do is add, say, "firstTransaction" to the action enum, and add a first_transaction_timing integer? No other extra data required?

@DLynch - Yes, that seems right. No extra data should be required.

@MNeisler - Sorry, I didn't think to ask -- for the timing, would it be more helpful if I stored time-since-initialization or time-since-ready? That'd sort of be whether this event is conceptually grouped with the editor-loading cluster or the saving cluster.

I have updated the schema for now, with time-since-ready: https://meta.wikimedia.org/w/index.php?title=Schema:EditAttemptStep&diff=19364181&oldid=19273145

The answer to that question will just require a documentation change if I picked incorrectly.

Change 535319 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/WikimediaEvents@master] Update EditAttemptStep

https://gerrit.wikimedia.org/r/535319

Change 535320 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/VisualEditor@master] Add new EditAttemptStep stage: firstTransaction

https://gerrit.wikimedia.org/r/535320

Change 535321 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/MobileFrontend@master] Add new EditAttemptStep stage: firstTransaction

https://gerrit.wikimedia.org/r/535321

Okay! That cluster of patches implements the firstTransaction step on desktop and mobile. It logs it with the timing as "time since the document was ready".

Change 535319 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Update EditAttemptStep

https://gerrit.wikimedia.org/r/535319

I assume this should be tracked in non-VE editors as well. In which case "firstTransaction" might be renamed "firstEdit" or "firstChange" as non VE surfaces don't necessarily use transactions. We should also write patches for those editors.

We should also write patches for those editors.

I did have that as a to-do, just lower priority than the initial round of VE testing.

Change 535599 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/WikimediaEvents@master] Update EditAttemptStep version

https://gerrit.wikimedia.org/r/535599

Change 535599 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Update EditAttemptStep version

https://gerrit.wikimedia.org/r/535599

Change 535612 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/WikiEditor@master] Implement firstChange logging

https://gerrit.wikimedia.org/r/535612

Okay, current patches implement firstChange in: desktop VisualEditor, desktop WikiEditor, MobileFrontend wikitext and visual.

Change 535321 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Add new EditAttemptStep stage: firstChange

https://gerrit.wikimedia.org/r/535321

Change 535320 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Add new EditAttemptStep stage: firstChange

https://gerrit.wikimedia.org/r/535320

Change 535612 merged by jenkins-bot:
[mediawiki/extensions/WikiEditor@master] Implement firstChange logging

https://gerrit.wikimedia.org/r/535612

Change 535218 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] MWSaveDialog: add more instrumentation

https://gerrit.wikimedia.org/r/535218

DLynch edited projects, added Skipped QA; removed Patch-For-Review.

T232790 is basically the QA process, after all.

dchan moved this task from Current work to Analysis on the VisualEditor board.Oct 6 2019, 6:09 AM
dchan edited projects, added VisualEditor; removed VisualEditor (Current work).

It looks like this is ready for Peter's review

ppelberg closed this task as Resolved.Oct 23 2019, 1:10 AM