Page MenuHomePhabricator

Add event tracking to the full-page editing button T409990 will introduce
Closed, ResolvedPublic

Description

T409990 will introduce a new button that will enable people who enter mobile section editing to edit the full article.

This task involves the work of adding the instrumentation necessary for us to analyze how people engage with this new button and what (if any) any resulting actions they take.

Decision(s) to be made

The instrumentation this ticket is asking for will enable the Editing Team to decide things like:

  • Will we prioritize work on a solution like T410096 people who tap the new Edit full page button are abandoning before the full-page reaches a ready state

Requirements

Meta

  • Tracking is added to the Edit full page button so that we can know things like:
    • What proportion of people who tap the Edit full page this ticket will introduce abandon before the full article is loaded in an editable state?
    • What proportion of people who tap the Edit full page this ticket will introduce go on to publish a constructive edit?

Event name(s)

Done

  • New event name(s) proposed
  • @MNeisler reviews event proposal(s)
  • New events instrumented
  • Editing QA verifies events being emitted in client
  • Product Analytics verifies events being logged server-side | @MNeisler

Event Timeline

Basic would be something like: feature: section-switch; action: switch.

We could expand it to contain more information though -- keep the feature, but I can imagine a range of actions: switch-top, switch-bottom. We could also log the section number you're switching from, which might be informative. Say switch-3-top etc.

I'm wondering if we should add mobile to the feature name to help clarify that this event does not occur on desktop. Maybe feature = section-switch-mobile?

Regarding actions, I think it would be useful to track some detail about button location and/or where the user was in the article when they switched. This would allow us to investigate if certain switch locations in the article result in less engagement by users or higher edit abandonment rate.

Logging the section number would be informative but might be too granular for this use case and make analytics a little complex. Another proposal would be to have an event sent for the three cases documented in T409990:

  • lead section (switch-lead)
  • middle section (switch-middle). *If we want info on button location, we could change this to switch-middle-top and switch-middle-bottom.
  • last section (switch-last)

This would give us a general sense of where in the article the user is clicking the button and how that might impact engagement and completion rates.

It's not technically mobile-exclusive -- it's anywhere that this section editing mode is enabled. Which is everywhere on mobile, and also wikitech and officewiki.

It's not technically mobile-exclusive -- it's anywhere that this section editing mode is enabled. Which is everywhere on mobile, and also wikitech and officewiki.

Got it, thanks for clarifying. Then I'm fine keeping as feature = section-switch.

I added a row to the instrumentation spec with a proposal of how this event can be collected in Test Kitchen using the web/base schema structure, which includes action and action_subtype fields.

action: section_switch
action_sub_type: (lead, middle, last) *to indicate switch location 
action_context: {
  "interface":  ###
   "timing_ms":###
}

Proposal for VEFU logging:

NameMeaning
event.feature = 'section-switch'User taps the "Edit full page" button after entering section editing mode.
event.action = 'switch-lead'User taps button presented at bottom of the lead section
event.action = 'switch-middle'User taps button presented at the top or bottom of middle section
event.action = 'switch-last'User taps button presented at the top of the last section

cc @ppelberg

@MNeisler: the proposal you shared in T410319#11402693 looks great to me.

@Esanders: what – if anything – about the way you've been implementing T409990 might need to be changed in order to implement the instrumentation proposal Megan outlined above (and copied below)?

Instrumentation proposal

NameMeaning
event.feature = 'section-switch'User taps the "Edit full page" button after entering section editing mode.
event.action = 'switch-lead'User taps button presented at bottom of the lead section
event.action = 'switch-middle'User taps button presented at the top or bottom of middle section
event.action = 'switch-last'User taps button presented at the top of the last section

The actions don't make much sense to me -- differentiating between lead/middle/last seems strange. Differentiating between top/bottom makes sense, but as-specified we wouldn't know that for the majority of sections (which are mostly middle, after all).

I think the goal should be to differentiate where we believe the user experience is different enough to potentially impact the identified metrics (e.g. edit abandonment rate).

I initially proposed lead/middle/last proposal to track the three UX cases described in T409990 and to provide an idea of user position (without being too granular). My thinking was that a person clicking the edit full page button from the lead section would have a different UX experience than someone editing a middle or last section and, as a result, we may see different trends in edit completion rates.

Differentiating between top/bottom makes sense, but as-specified we wouldn't know that for the majority of sections (which are mostly middle, after all).

Yeah, we would not have data on whether a top or bottom edit full page button was clicked for middle section. I'm fine adding this detail if we think button location might impact our identified metrics or if knowing which button is clicked more might help inform future UX design decisions.

One option would be to revise the event.action = 'switch-middle' into the following?:

  • event.action = 'switch-middle-top'
  • event.action = 'switch-middle-bottom'

Change #1213576 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@master] Add instrumentation for mobile section switching

https://gerrit.wikimedia.org/r/1213576

Change #1213576 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Add instrumentation for mobile section switching

https://gerrit.wikimedia.org/r/1213576

ppelberg moved this task from Inbox to High Priority on the Editing QA board.

Change #1216608 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@wmf/1.46.0-wmf.5] Add instrumentation for mobile section switching

https://gerrit.wikimedia.org/r/1216608

Change #1216608 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@wmf/1.46.0-wmf.5] Add instrumentation for mobile section switching

https://gerrit.wikimedia.org/r/1216608

Mentioned in SAL (#wikimedia-operations) [2025-12-08T21:16:47Z] <urbanecm@deploy2002> Started scap sync-world: Backport for [[gerrit:1216608|Add instrumentation for mobile section switching (T410319)]], [[gerrit:1216609|Edit full page: Tweak skeleton appearance and fix scroll offsets]], [[gerrit:1216611|Set full page scroll to 130px (T411669)]], [[gerrit:1216612|Ensure images are fixed size on mobile while loading (T411669)]]

Mentioned in SAL (#wikimedia-operations) [2025-12-08T21:18:40Z] <urbanecm@deploy2002> kemayo, urbanecm: Backport for [[gerrit:1216608|Add instrumentation for mobile section switching (T410319)]], [[gerrit:1216609|Edit full page: Tweak skeleton appearance and fix scroll offsets]], [[gerrit:1216611|Set full page scroll to 130px (T411669)]], [[gerrit:1216612|Ensure images are fixed size on mobile while loading (T411669)]] synced to the testservers (see https://wikitech.wikimedia.org/w

Mentioned in SAL (#wikimedia-operations) [2025-12-08T21:24:24Z] <urbanecm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1216608|Add instrumentation for mobile section switching (T410319)]], [[gerrit:1216609|Edit full page: Tweak skeleton appearance and fix scroll offsets]], [[gerrit:1216611|Set full page scroll to 130px (T411669)]], [[gerrit:1216612|Ensure images are fixed size on mobile while loading (T411669)]] (duration: 07m 36s)

ppelberg reassigned this task from Somebody to EAkinloose.
ppelberg added a subscriber: Somebody.

User taps the "Edit full page" button after entering section editing mode
User taps button presented at bottom of the lead section

{
    "wiki": "enwiki",
    "skin": "minerva",
    "is_bot": true,
    "feature": "section-switch",✅
    "action": "switch-lead-bottom", ✅
    "editor_interface": "visualeditor",
    "integration": "page",
    "platform": "phone",
    "editingSessionId": "***",
    "is_oversample": false,
    "bucket": "2025-09-editcheck-addReference-test"
}

User taps button presented at the top or bottom of middle section

{
    "wiki": "enwiki",
    "skin": "minerva",
    "is_bot": false,
    "feature": "section-switch", ✅
    "action": "switch-middle-top", ✅
    "editor_interface": "visualeditor",
    "integration": "page",
    "platform": "phone",
    "editingSessionId": "***",
    "is_oversample": false,
    "bucket": "2025-09-editcheck-addReference-control"
}
{
    "wiki": "enwiki",
    "skin": "minerva",
    "is_bot": true,
    "feature": "section-switch", ✅
    "action": "switch-middle-bottom", ✅
    "editor_interface": "visualeditor",
    "integration": "page",
    "platform": "phone",
    "editingSessionId": "***",
    "is_oversample": false,
    "bucket": "2025-09-editcheck-addReference-test"
}

User taps button presented at the top of the last section

{
    "wiki": "enwiki",
    "skin": "minerva",
    "is_bot": true,
    "feature": "section-switch",✅
    "action": "switch-last-top",✅
    "editor_interface": "visualeditor",
    "integration": "page",
    "platform": "phone",
    "editingSessionId": "***",
    "is_oversample": false,
    "bucket": "2025-09-editcheck-addReference-test"
}

❌ Console error

image.png (1×3 px, 857 KB)

To reproduce, tap button presented at the top of the *last section*

@ppelberg
I've confirmed these events are logging in VisualEditorFeatureUse as expected. See summary of checks and some initial data below:

  • There have been 191 editing sessions with at least one click to the edit full page button (feature = section-switch)
    • 124 editing sessions from logged out editors; 67 editing sessions from logged in editors
  • Confirmed events are only logged on mobile web visual editor as expected.
  • Events have been recorded for all four actions implemented to indicate button location and type.
    • So far, the edit full page button shown in the lead section is selected the most (action = switch-lead-bottom) .
Edit full page button type% of editing sessions with edit full page button click
switch-lead-bottom75%
switch-middle-bottom21%
switch-middle-top3.7%
switch-last-top0.5%