Page MenuHomePhabricator

Instrument the Tone Check UX
Closed, ResolvedPublic

Description

This ticket involves the work of implementing the instrumentation required to track how people are engaging with the Tone Check UX.

Per offline discussion, the instrumentation required to do the above was added as part of the Multi-Check feature implementation (T352092).

As such, this ticket largely holds the work of verifying the ===Requirements have been implemented as expected.

Requirements

featureactionmeaning
editCheck-toneaction-editUser clicks "revise" button to accept tone check
editCheck-toneaction-dismissUser clicks "decline" button to reject tone check
editCheck-tonecheck-shown-[moment]Indicates whether the check was shown pre-save or mid-edit. An event will be sent for every tone check shown.
editCheckDialogwindow-open-from-check-[moment]Indicates when the sidebar to present edit checks was shown

Progress

Next steps

  • Megan to finish populating requirements
  • David to confirm instrumentation is implemented
  • Esther to QA
  • Megan to QA

Done

  • Editing QA verifies ===Requirements are being logged client-side
  • @MNeisler verifies ===Requirements are being logged server-side

Event Timeline

Aklapper renamed this task from Instrument the Peacock Check UX to Instrument the Tone Check UX.May 28 2025, 11:43 AM
ppelberg updated the task description. (Show Details)
ppelberg added a subscriber: MNeisler.
ppelberg added a subscriber: DLynch.

@DLynch: would it be accurate for Megan and I to think the Tone Check instrumentation is deployed and available for testing via ecenable=2?

The ones described in this ticket should be available that way, yes.

The ones described in this ticket should be available that way, yes.

Excellent, ok. Thank you for confirming, David.

Next steps

  • Megan to finish populating requirements
  • David to confirm instrumentation is implemented
  • Esther to QA
  • Megan to QA
MNeisler added subscribers: EAkinloose, DavidL.

I've finished populating requirements in the task description.

Assigning over to @EAkinloose as I believe this is also ready for QA (cc @DLynch to confirm)

Had to update the actions for the button-presses, but this should now be testable as-described.

ppelberg moved this task from Inbox to High Priority on the Editing QA board.
ppelberg moved this task from Inbox to QA on the Editing-team (Kanban Board) board.
ppelberg updated the task description. (Show Details)

✅ ===Requirements are being logged client-side

  • editCheck-tone -> check-shown-[moment]:
{
    ...
    "is_bot": false,
    "feature": "editCheck-tone",
    "action": "check-shown-presave",
    "editor_interface": "visualeditor",
    ...
}

{
    ...
    "is_bot": false,
    "feature": "editCheck-tone",
    "action": "check-shown-midedit",
    "editor_interface": "visualeditor",
    ...
}
  • editCheck-tone -> action-dismiss:
{
    ...
    "is_bot": false,
    "feature": "editCheck-tone",
    "action": "action-dismiss",
    "editor_interface": "visualeditor",
    ...
}
  • editCheckDialog -> window-open-from-check-[moment]:
{
    ...
    "is_bot": false,
    "feature": "editCheckDialog",
    "action": "window-open-from-check-midedit",
    "editor_interface": "visualeditor",
    ...
}
  • editCheck-tone -> action-edit:

@EAkinloose: can you please confirm what wiki you tested this on? This information will help Megan verify events are being emitted and logged as expected.

@EAkinloose: can you please confirm what wiki you tested this on? This information will help Megan verify events are being emitted and logged as expected.

en-beta. https://en.wikipedia.beta.wmflabs.org/w/index.php?title=Basic_example_test_sheet&veaction=edit&ecenable=2&trackdebug=1

@EAkinloose: can you please confirm what wiki you tested this on? This information will help Megan verify events are being emitted and logged as expected.

en-beta. https://en.wikipedia.beta.wmflabs.org/w/index.php?title=Basic_example_test_sheet&veaction=edit&ecenable=2&trackdebug=1

Wonderful – thank you for confirming, Esther.

I ran a quick check to verify events to track tone check engagement are being stored as expected in VisualEditorFeatureUse. This is based on a review of test events that have been generated on various wikis since events were implemented through 10 July 2025.

See summary of the checks completed below:

  • 13 test editing sessions logged on enwiki, frwiki, and ptwiki so far. 46 total distinct tone check events recorded.
  • All expected tone check events are logged including: action-dismiss, action-edit, check-shown-midedit, check-shown-presave, click-learn-more.
  • Events are correctly associated with the editCheck-tone feature.
  • The most frequently logged event is check-shown-midedit. There have been 25 check-shown-midedit eents and only 1 check-shown-presave event so far based on test events.
  • All events are recorded as occurring with visualeditor (editor_interface = 'visualeditor')
  • Tone check events are logged for both platform and mobile platforms
  • Tone Check events are logged for both unregistred and registered users.
  • The order of events within a session appears as expected (i.e. check-shown-midedit appears before action-edit)

I've updated the VEFU data dictionary so these new tone check events are now reflected there.

Note: QA to confirm bucketing is balanced and bucketing specific fields are logging as expected will occur in T395090

@MNeisler – this review looks good to me. Closing this out. Thank you!