Page MenuHomePhabricator

Track usage of the "open here to edit" feature
Closed, ResolvedPublic

Description

In 6.2.1 the app added the ability to select (almost) any text and tap a contextual menu item to "Edit", which takes the user to that piece of content in the edit view.

We'd like to track usage of this feature, specifically:

  • Absolute count of times someone tapped Edit from the selection menu
  • Number of unique users tapping the Edit

Some potential "nice to have":

  • Times the selection menu was show, but the user didn't tap edit
  • Edit/Save completion rates for users/events where they got to the editor via this affordance. Compare with conversion rate when user start editing from the edit pencil. Currently, the edit funnel looks like:
First User will open the editor > if it’s their first time they see onboarding
                                > if it’s not their first time, they go straight to the editor > make edits > next button > see the preview > next button > edit summary > publish

Zeplin: https://app.zeplin.io/project/57a120ce9787dcf26230651f


To answer the questions above, we want to make the following changes to the edit funnel (MobileWikiAppEdit):

  • Add a "source" field to every event, identifying where the edit was invoked: tap on title description (title description), pencil tap (pencil), or contextual menu tap (highlight).
  • Add action "onboarding" to the edit funnel.
  • Add action "ready" and "editSummaryShown" to the edit funnel for Wikipedia edit.
  • If T217554 is fixed, replace errorText with unlocalized codes.
  • Revision: 18115551 -> 19082700 (Please ignore the revision number in the example JSON below)

Title description edit

All Wikidata descriptions editing events should contains the following fields: action, wikidataDescriptionEdit, source, anon, app_install_id, session_token, client_dt. If the edit is saved successfully, we record the revision number in the revID field with the action=saved. If there is any error, we record the unlocalized codes (if T217554 is fixed), or the error message returned in the errorText field with the action=error.

Tap on "Add title description"Tap on pencil, then "Edit title description"Highlight title description, then tap on "Edit"Onboarding shown for new editorsReady to edit (cursor blinking)Attempt to saveSaved successfullyError (if any)

Wikipedia edit

All Wikipedia editing events should contains the following fields: action, source, anon, app_install_id, session_token, client_dt. If the edit is saved successfully, we record the revision number in the revID field with the action=saved. If there is any error, we record the unlocalized codes (if T217554 is fixed), or the error message returned in the errorText field with the action=error.

Tap on pencil (If it's the introduction part, then tap on "Edit introduction")Highlighting text, then selecting 'edit' from the highlight menuOnboarding shown for new editorsReady to edit (cursor blinking or text highlighted)PreviewEdit summaryTap on "Publish" (Attempt to save)Saved successfullyError (if any)

Event Timeline

JMinor created this task.Mar 21 2019, 7:22 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 21 2019, 7:22 PM
JMinor triaged this task as Medium priority.Mar 25 2019, 6:40 PM
JMinor moved this task from Needs Triage to Product Backlog on the Wikipedia-iOS-App-Backlog board.

Edit/Save completion rates for users/events where they got to the editor via this affordance

Hi @cmadeo, Josh and I want to compare the drop-off rate of users from start edit to save for different channels, i.e. start from the pencil sign, start from "open here to edit" feature, and others (if any). I'm wondering if you have flow charts for these different channels. And can I schedule a meeting with you to talk about it? Thanks!

@chelsyx I'll pull together some flows for our meeting tomorrow!

chelsyx updated the task description. (Show Details)Mar 28 2019, 9:05 PM
chelsyx updated the task description. (Show Details)Mar 28 2019, 9:08 PM
chelsyx added a project: Product-Analytics.
chelsyx moved this task from Triage to Next Up on the Product-Analytics board.
chelsyx moved this task from Next Up to Doing on the Product-Analytics board.Apr 24 2019, 6:45 PM
chelsyx updated the task description. (Show Details)May 6 2019, 2:21 AM
chelsyx updated the task description. (Show Details)May 6 2019, 4:02 AM

Hi @JMinor, for the "open here to edit" feature, when users highlight a non-editable string and tap "edit", an alert would show up with the edit interface open (T215448). I didn't track this event in the current schema, but please let me know if you think it's necessary to track this behavior.

chelsyx updated the task description. (Show Details)May 7 2019, 6:55 PM
chelsyx updated the task description. (Show Details)May 7 2019, 7:01 PM

Per our discussion, the initial metrics woun't explicitly track non-editable taps or attempts to edit non-editable content. I've created a sub task for this if/when we want to add it in the next iteration, but for now, this schema/plan as is acceptable.

chelsyx added a comment.EditedJun 11 2019, 6:24 AM

Hi @NHarateh_WMF ! I have a question: I noticed there is a revID associated with every editing event -- from start, ready to saved successfully. To my understanding, the revID is returned by the API only when the edit is saved successfully, is that right?

@chelsyx every article has a revision ID - it changes when an article changes. Do you want to log it only when it changes (edit saved)?

@chelsyx every article has a revision ID - it changes when an article changes. Do you want to log it only when it changes (edit saved)?

@NHarateh_WMF I just want to make sure the revision ID we logged is the same as the revision ID in Mediawiki. I'm not sure whether the app is using this API, but we want to log the newrevid in this response:

{
        "edit":{
        "result":"Success",
        "pageid":94542,
        "title":"Sandbox",
        "contentmodel":"wikitext",
        "oldrevid":371705,
        "newrevid":371707,
        "newtimestamp":"2018-12-18T16:59:42Z"
    }
}

Sounds good @chelsyx, I'll log the newrevid with success events and drop the current id from other events 👍

Hi @NHarateh_WMF, I've finished the QA. Besides the issue of revID, the other thing we need to fix is when edit Wikipedia articles, the action of preview and editSummaryShown are not sent:

PreviewEdit summary

See the testing scenarios spreadsheet for more details: https://docs.google.com/spreadsheets/d/1MgXn6zR-YKaxW_5kXWcj0vcuIq8Twjdopr2HKub6Hhc/edit?usp=sharing

@chelsyx thanks for testing! It should be fixed when this gets merged https://github.com/wikimedia/wikipedia-ios/pull/3124
Let me know if you come across any other bugs

Thanks @NHarateh_WMF ! I just confirmed that this patch (https://github.com/wikimedia/wikipedia-ios/pull/3124) fixed all the bugs.

JMinor added a subscriber: NHarateh_WMF.

Please recheck data once release has been out for a couple weeks.

@chelsyx will do some knowledge transfer with @Mayakp.wiki when Maya is in the office next week

Just checked the database. It all looks good, except that for preview action, a source field identifying whether users start from pencil button or highlight is missing. @NHarateh_WMF can you help fix it? Thanks!

Preview
chelsyx removed chelsyx as the assignee of this task.Jul 18 2019, 5:22 AM
ABorbaWMF added a subscriber: ABorbaWMF.

Looks ok to me from a UI standpoint on 6.3.1 (1649). It appears the actual tracking has also been verified. Moving to sign off.

JMinor closed this task as Resolved.Aug 14 2019, 10:22 PM