Page MenuHomePhabricator

Track usage of the "open here to edit" feature
Closed, ResolvedPublic

Description

In 6.2.1 the app added the ability to select (almost) any text and tap a contextual menu item to "Edit", which takes the user to that piece of content in the edit view.

We'd like to track usage of this feature, specifically:

  • Absolute count of times someone tapped Edit from the selection menu
  • Number of unique users tapping the Edit

Some potential "nice to have":

  • Times the selection menu was show, but the user didn't tap edit
  • Edit/Save completion rates for users/events where they got to the editor via this affordance. Compare with conversion rate when user start editing from the edit pencil. Currently, the edit funnel looks like:
First User will open the editor > if it’s their first time they see onboarding
                                > if it’s not their first time, they go straight to the editor > make edits > next button > see the preview > next button > edit summary > publish

Zeplin: https://app.zeplin.io/project/57a120ce9787dcf26230651f


To answer the questions above, we want to make the following changes to the edit funnel (MobileWikiAppEdit):

  • Add a "source" field to every event, identifying where the edit was invoked: tap on title description (title description), pencil tap (pencil), or contextual menu tap (highlight).
  • Add action "onboarding" to the edit funnel.
  • Add action "ready" and "editSummaryShown" to the edit funnel for Wikipedia edit.
  • If T217554 is fixed, replace errorText with unlocalized codes.
  • Revision: 18115551 -> 19082700 (Please ignore the revision number in the example JSON below)

Title description edit

All Wikidata descriptions editing events should contains the following fields: action, wikidataDescriptionEdit, source, anon, app_install_id, session_token, client_dt. If the edit is saved successfully, we record the revision number in the revID field with the action=saved. If there is any error, we record the unlocalized codes (if T217554 is fixed), or the error message returned in the errorText field with the action=error.

Tap on "Add title description"Tap on pencil, then "Edit title description"Highlight title description, then tap on "Edit"Onboarding shown for new editorsReady to edit (cursor blinking)Attempt to saveSaved successfullyError (if any)
01 Wikidata descriptions.png (1×750 px, 744 KB)
wikidata edit start.png (1×828 px, 239 KB)
Simulator Screen Shot - iPhone X - 2019-05-05 at 18.49.43.png (2×1 px, 873 KB)
02 Wikidata descriptions.png (1×750 px, 78 KB)
04 Wikidata descriptions.png (1×750 px, 120 KB)
05 Wikidata descriptions.png (1×750 px, 128 KB)
07 Wikidata descriptions.png (1×750 px, 331 KB)
IMG_2324.PNG (1×750 px, 110 KB)
Screen Shot 2019-05-05 at 7.04.50 PM.png (466×984 px, 74 KB)
Screen Shot 2019-05-05 at 7.08.11 PM.png (470×972 px, 73 KB)
Screen Shot 2019-05-05 at 7.09.15 PM.png (460×988 px, 74 KB)
Screen Shot 2019-05-05 at 7.09.54 PM.png (454×970 px, 78 KB)
Screen Shot 2019-05-05 at 7.10.21 PM.png (474×964 px, 78 KB)
Screen Shot 2019-05-05 at 7.10.51 PM.png (454×964 px, 79 KB)
Screen Shot 2019-05-05 at 7.18.11 PM.png (496×978 px, 84 KB)
Screen Shot 2019-05-05 at 7.11.44 PM.png (484×1 px, 99 KB)

Wikipedia edit

All Wikipedia editing events should contains the following fields: action, source, anon, app_install_id, session_token, client_dt. If the edit is saved successfully, we record the revision number in the revID field with the action=saved. If there is any error, we record the unlocalized codes (if T217554 is fixed), or the error message returned in the errorText field with the action=error.

Tap on pencil (If it's the introduction part, then tap on "Edit introduction")Highlighting text, then selecting 'edit' from the highlight menuOnboarding shown for new editorsReady to edit (cursor blinking or text highlighted)PreviewEdit summaryTap on "Publish" (Attempt to save)Saved successfullyError (if any)
Simulator Screen Shot - iPhone X - 2019-05-05 at 20.08.46.png (2×1 px, 1 MB)
37 Highlight to edit.png (1×750 px, 344 KB)
Simulator Screen Shot - iPhone X - 2019-05-05 at 17.30.47.png (2×1 px, 166 KB)
38 Highlight to edit.png (1×750 px, 207 KB)
31 Preview.png (1×750 px, 342 KB)
32 Edit summary not logged in.png (1×750 px, 122 KB)
32 Edit summary not logged in.png (1×750 px, 123 KB)
34 - Edit confirmation.png (1×750 px, 208 KB)
Simulator Screen Shot - iPhone X - 2019-05-05 at 20.40.14.png (2×1 px, 204 KB)
Screen Shot 2019-05-05 at 8.54.41 PM.png (418×970 px, 66 KB)
Screen Shot 2019-05-05 at 8.55.10 PM.png (418×964 px, 66 KB)
Screen Shot 2019-05-05 at 8.56.03 PM.png (422×960 px, 69 KB)
Screen Shot 2019-05-05 at 8.56.34 PM.png (426×960 px, 69 KB)
Screen Shot 2019-05-05 at 8.57.01 PM.png (420×960 px, 69 KB)
Screen Shot 2019-05-05 at 8.57.26 PM.png (422×962 px, 68 KB)
Screen Shot 2019-05-05 at 8.57.54 PM.png (422×960 px, 69 KB)
Screen Shot 2019-05-05 at 8.58.18 PM.png (446×958 px, 72 KB)
Screen Shot 2019-05-05 at 8.58.51 PM.png (450×1 px, 87 KB)

Event Timeline

JMinor triaged this task as Medium priority.Mar 25 2019, 6:40 PM
JMinor moved this task from Needs Triage to Product Backlog on the Wikipedia-iOS-App-Backlog board.

Edit/Save completion rates for users/events where they got to the editor via this affordance

Hi @cmadeo, Josh and I want to compare the drop-off rate of users from start edit to save for different channels, i.e. start from the pencil sign, start from "open here to edit" feature, and others (if any). I'm wondering if you have flow charts for these different channels. And can I schedule a meeting with you to talk about it? Thanks!

@chelsyx I'll pull together some flows for our meeting tomorrow!

Hi @JMinor, for the "open here to edit" feature, when users highlight a non-editable string and tap "edit", an alert would show up with the edit interface open (T215448). I didn't track this event in the current schema, but please let me know if you think it's necessary to track this behavior.

Per our discussion, the initial metrics woun't explicitly track non-editable taps or attempts to edit non-editable content. I've created a sub task for this if/when we want to add it in the next iteration, but for now, this schema/plan as is acceptable.

Hi @NHarateh_WMF ! I have a question: I noticed there is a revID associated with every editing event -- from start, ready to saved successfully. To my understanding, the revID is returned by the API only when the edit is saved successfully, is that right?

@chelsyx every article has a revision ID - it changes when an article changes. Do you want to log it only when it changes (edit saved)?

@chelsyx every article has a revision ID - it changes when an article changes. Do you want to log it only when it changes (edit saved)?

@NHarateh_WMF I just want to make sure the revision ID we logged is the same as the revision ID in Mediawiki. I'm not sure whether the app is using this API, but we want to log the newrevid in this response:

{
        "edit":{
        "result":"Success",
        "pageid":94542,
        "title":"Sandbox",
        "contentmodel":"wikitext",
        "oldrevid":371705,
        "newrevid":371707,
        "newtimestamp":"2018-12-18T16:59:42Z"
    }
}

Sounds good @chelsyx, I'll log the newrevid with success events and drop the current id from other events 👍

Hi @NHarateh_WMF, I've finished the QA. Besides the issue of revID, the other thing we need to fix is when edit Wikipedia articles, the action of preview and editSummaryShown are not sent:

PreviewEdit summary
31 Preview.png (1×750 px, 342 KB)
32 Edit summary not logged in.png (1×750 px, 122 KB)
Screen Shot 2019-05-05 at 8.57.01 PM.png (420×960 px, 69 KB)
Screen Shot 2019-05-05 at 8.57.26 PM.png (422×962 px, 68 KB)

See the testing scenarios spreadsheet for more details: https://docs.google.com/spreadsheets/d/1MgXn6zR-YKaxW_5kXWcj0vcuIq8Twjdopr2HKub6Hhc/edit?usp=sharing

@chelsyx thanks for testing! It should be fixed when this gets merged https://github.com/wikimedia/wikipedia-ios/pull/3124
Let me know if you come across any other bugs

JMinor added a subscriber: NHarateh_WMF.

Please recheck data once release has been out for a couple weeks.

@chelsyx will do some knowledge transfer with @Mayakp.wiki when Maya is in the office next week

Just checked the database. It all looks good, except that for preview action, a source field identifying whether users start from pencil button or highlight is missing. @NHarateh_WMF can you help fix it? Thanks!

Preview
31 Preview.png (1×750 px, 342 KB)
Screen Shot 2019-05-05 at 8.57.01 PM.png (420×960 px, 69 KB)
ABorbaWMF added a subscriber: ABorbaWMF.

Looks ok to me from a UI standpoint on 6.3.1 (1649). It appears the actual tracking has also been verified. Moving to sign off.