Page MenuHomePhabricator

VE mobile default: start tracking editor switches on mobile + desktop
Closed, ResolvedPublic

Description

"Done"

  • We are now tracking when contributors on mobile and desktop switch from one editing interface to another

Event Timeline

ppelberg created this task.Apr 17 2019, 6:26 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 17 2019, 6:26 AM
ppelberg updated the task description. (Show Details)Apr 17 2019, 6:29 AM
ppelberg updated the task description. (Show Details)
ppelberg renamed this task from Implement instrumentation for switching between editors to VE mobile default: implement instrumentation for switching between editors.Apr 17 2019, 7:18 AM
ppelberg updated the task description. (Show Details)
marcella removed ppelberg as the assignee of this task.May 15 2019, 4:09 PM
ppelberg renamed this task from VE mobile default: implement instrumentation for switching between editors to VE mobile default: start tracking editor switches on mobile + desktop.Jun 24 2019, 8:03 PM
ppelberg assigned this task to DLynch.
ppelberg updated the task description. (Show Details)
ppelberg updated the task description. (Show Details)

Change 519281 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/MobileFrontend@master] Log editor switches to visualeditorfeatureuse

https://gerrit.wikimedia.org/r/519281

ppelberg updated the task description. (Show Details)Jun 26 2019, 9:10 PM

Change 519281 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Log editor switches to visualeditorfeatureuse

https://gerrit.wikimedia.org/r/519281

That's mobile done. Another patch for desktop will be forthcoming.

@DLynch how is this different from logging an abort_type of switchwith or switchwithout? Admittedly, that doesn't tell us which editor the user is switching to, but that's generally easy to infer.

Change 519498 had a related patch set uploaded (by Jforrester; owner: DLynch):
[mediawiki/extensions/MobileFrontend@wmf/1.34.0-wmf.11] Log editor switches to visualeditorfeatureuse

https://gerrit.wikimedia.org/r/519498

@Neil_P._Quinn_WMF That's something which has been subject to general code-atrophy over time. It only fires the abort for a full teardown editor-switch on desktop (so NWE isn't counted), and mobile has gradually wound up getting in-line with that so it's not logging the aborts because it's not navigating away. I figured putting in a consistent "this is feature-usage, and not an abstract thing tied to page lifecycle" would get us a less fragile metric going forward.

Change 519498 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@wmf/1.34.0-wmf.11] Log editor switches to visualeditorfeatureuse

https://gerrit.wikimedia.org/r/519498

Mentioned in SAL (#wikimedia-operations) [2019-06-27T20:31:19Z] <jforrester@deploy1001> Synchronized php-1.34.0-wmf.11/extensions/MobileFrontend/resources/dist: T221191: Log editor switches to visualeditorfeatureuse (duration: 00m 50s)

Change 519529 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/VisualEditor@master] Log editor switches to visualeditorfeatureuse

https://gerrit.wikimedia.org/r/519529

Change 519530 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/WikiEditor@master] Log editor switches to visualeditorfeatureuse

https://gerrit.wikimedia.org/r/519530

I found an extra little data-glitch in there -- when switching from WikiEditor to VisualEditor it wouldn't log it properly, because it wasn't updated to correctly see the in-page load. It would thus wait to log an abort until VE was done and triggered a page-reload, and it wouldn't log that abort correctly as being a switch away from WikiEditor (because it detected that purely based on whether the link to visual editing currently had focus).

So, @Neil_P._Quinn_WMF: all analytics data about switching to VE from WikiEditor from the last 3-ish years is suspect. 😁

Neil_P._Quinn_WMF added a comment.EditedJul 1 2019, 8:11 PM

I found an extra little data-glitch in there -- when switching from WikiEditor to VisualEditor it wouldn't log it properly, because it wasn't updated to correctly see the in-page load. It would thus wait to log an abort until VE was done and triggered a page-reload, and it wouldn't log that abort correctly as being a switch away from WikiEditor (because it detected that purely based on whether the link to visual editing currently had focus).
So, @Neil_P._Quinn_WMF: all analytics data about switching to VE from WikiEditor from the last 3-ish years is suspect. 😁

Hahaha, wow, multiple good catches! This will definitely help us tracking switching behavior during the A/B test 👏

Out of curiosity, I checked the percentage of aborts in each editor that were switches, and it lines up with what you say (where desktop VE is the only editor logging any at all).

select
    event.platform as platform,
    event.editor_interface as editor,
    concat(round((
        sum(cast(event.abort_type like "switch%" as int)) * 100 / count(*)
    ), 1), "%") as switch_aborts
-- From the unsanitized data, so it only covers the last 90 days
from event.editattemptstep
where
    event.action = "abort" and
    -- Partition predicate to make Hive happy
    year > 0
group by event.platform, event.editor_interface
platformeditorswitch_aborts
phonevisualeditor0.0%
desktopvisualeditor9.3%
phonewikitext0.0%
desktopwikitext-20170.0%
desktopwikitext0.0%

Change 519529 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Log editor switches to visualeditorfeatureuse

https://gerrit.wikimedia.org/r/519529

Change 519530 merged by jenkins-bot:
[mediawiki/extensions/WikiEditor@master] Log editor switches to visualeditorfeatureuse

https://gerrit.wikimedia.org/r/519530

ppelberg closed this task as Resolved.Sep 3 2019, 1:42 AM