Page MenuHomePhabricator

Instrument mobile wikitext editor fallback worfklow
Open, Needs TriagePublic

Description

Background

The deployment of T239676 will introduce an option for all contributors, including those included in our ongoing A/B test [1], to opt into the mobile wikitext editor, even in instances when the mobile visualeditor would have otherwise loaded by default. [2]

To ensure these contributors are appropriately accounted for in our analysis, we will need to add new tracking events that enable us to see which users assigned to the default-visual bucket have opted into the mobile wikitext editor by way of the new "fallback workflow" T239676 will introduce.

Open questions

What we would like to know

We would like to know...

  • When/whether a contributor is shown the fallback prompt
  • Whether they elect to use the fallback editor
    • How long after being shown the fallback editor do they elect to use the fallback editor
    • When a contributor elects to use the fallback editor whether they switch to visual mode
  • Which contributors in the A/B test group were shown the fallback prompt

Done

  • All "Open questions" above have been resolved
  • The instrumentation required to satisfy the "What we would like to know" requirements have been defined
  • The instrumentation required to satisfy the "What we would like to know" requirements have been implemented

  1. T227338: Test visual editor as the default mobile editor on select wikis
  2. E.g. a contributor in the A/B test could be assigned to the default-visual bucket, but elect to use the mobile wikitext editor if they felt the mobile visual editor was taking too long to load.

Event Timeline

ppelberg updated the task description. (Show Details)Jan 25 2020, 12:02 AM
matmarex added a subscriber: matmarex.EditedJan 25 2020, 3:48 PM

I noted in the commit message of https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MobileFrontend/+/554222:

We […] log an init event with
init_mechanism=tooslow.

Is this enough? If not, what would you like?

Mayakp.wiki moved this task from Triage to Tracking on the Product-Analytics board.Feb 3 2020, 5:34 PM
matmarex moved this task from To Triage to Triaged on the VisualEditor board.Feb 12 2020, 5:21 PM
ppelberg updated the task description. (Show Details)Feb 15 2020, 12:24 AM

I noted in the commit message of https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MobileFrontend/+/554222:

We […] log an init event with
init_mechanism=tooslow.

Is this enough? If not, what would you like?

This instrumentation should be helpful – thank you, Bartosz. I've added what we would like to know about how people engage with the fallback experience to the task description.

If you have thoughts about either of the above, please add them here.

Next steps

  • @Mayakp.wiki to review "What we would like to know" section in the task description and determine what – if any – additional instrumentation we will need to implement.
    • Note: I do not anticipate Maya having time to do this for two weeks, considering all we have going on with the instrumentation around the Replying workflow (T243364)