Page MenuHomePhabricator

Add instrumentation to VEFU to detect when content is pasted within an edit session
Closed, ResolvedPublic

Description

This task involves the work of adding instrumentation to visual editor feature use that will enable us to identify published edits that Paste Check would have been shown within had the feature been enabled.

The need for this instrumentation emerged in response to the Editing Team deciding not to track this information using edit tags, as we've done for Reference Check and Tone Check. Please see T379843#11104424 for info. on how this decision came to be.

Stories

As the person responsible for evaluating the impact of Paste Check, I need to be able to identify edit attempts in which Paste Check would have been shown Paste Check during an edit session (were it enabled) so that I can determine the impact of Paste Check's presence/absence on edit quality.

Requirements

  • Event sent for all edit attempts that would have caused Paste Check to be shown at least once in an edit session were it available.
    • Note: this instrumentation will need to be kept in sync with how Paste Check is configured on a per project basis
  • Event logged for each moment Paste Check would have been shown to users bucketed in the control group.
    • Note: This should be similar to the feature: editCheck-tone, action: check-control-[listener] implemented for Tone Check in T394952.
  • Events logged in VisualEditorFeatureUse.

Done

  • @MNeisler verifies Requirements
  • Eng implements Requirements
  • Editing QA verifies event is being emitted client-side
  • Product Analytics verifies event is being logged server-side

Event Timeline

MNeisler edited projects, added Product-Analytics; removed Product-Analytics (Kanban).

I've updated and verified the requirements. Reassigning to @DLynch for review and implementation.

MNeisler moved this task from Triage to Tracking on the Product-Analytics board.

Change #1186636 had a related patch set uploaded (by DLynch; author: DLynch):

[VisualEditor/VisualEditor@master] ce.ClipboardHandler: emit a ce.Surface#paste event afterPaste

https://gerrit.wikimedia.org/r/1186636

Change #1186638 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@master] Paste check A/B: log when a paste check would be shown if in control group

https://gerrit.wikimedia.org/r/1186638

This will only log something when the a/b test is enabled and the user is in the control group. It will log feature: editCheck-paste, action: check-control-[listener]... and because you can only paste stuff when in mid-edit, in practice it'll always be action: check-control-onDocumentChange.

Change #1187035 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@master] Paste check: rename for consistency

https://gerrit.wikimedia.org/r/1187035

This will now log feature: editCheck-paste, action: relevant-paste at any time the paste check is not-enabled (no a/b test, or control group). As soon as the paste check is enabled, it will stop logging, and you'll need to fall back on the feature: editCheck-paste, action: check-shown-[moment] event.

Change #1186636 merged by jenkins-bot:

[VisualEditor/VisualEditor@master] ce.ClipboardHandler: emit a ce.Surface#paste event afterPaste

https://gerrit.wikimedia.org/r/1186636

Change #1188403 had a related patch set uploaded (by Esanders; author: Esanders):

[mediawiki/extensions/VisualEditor@master] Update VE core submodule to master (70eee46f0)

https://gerrit.wikimedia.org/r/1188403

Change #1188403 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Update VE core submodule to master (70eee46f0)

https://gerrit.wikimedia.org/r/1188403

Change #1187035 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Paste check: rename for consistency

https://gerrit.wikimedia.org/r/1187035

Change #1186638 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Paste check: log when a paste check would have been shown if enabled

https://gerrit.wikimedia.org/r/1186638

ppelberg moved this task from Code Review to QA on the Editing-team (Kanban Board) board.
ppelberg added a project: Editing QA.
ppelberg moved this task from Inbox to High Priority on the Editing QA board.

Change #1189374 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@wmf/1.45.0-wmf.19] Paste check: log when a paste check would have been shown if enabled

https://gerrit.wikimedia.org/r/1189374

Change #1189374 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@wmf/1.45.0-wmf.19] Paste check: log when a paste check would have been shown if enabled

https://gerrit.wikimedia.org/r/1189374

Mentioned in SAL (#wikimedia-operations) [2025-09-17T23:24:48Z] <kemayo@deploy1003> Started scap sync-world: Backport for [[gerrit:1189374|Paste check: log when a paste check would have been shown if enabled (T402460)]]

Mentioned in SAL (#wikimedia-operations) [2025-09-17T23:31:09Z] <kemayo@deploy1003> kemayo: Backport for [[gerrit:1189374|Paste check: log when a paste check would have been shown if enabled (T402460)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-09-17T23:37:26Z] <kemayo@deploy1003> Finished scap sync-world: Backport for [[gerrit:1189374|Paste check: log when a paste check would have been shown if enabled (T402460)]] (duration: 12m 38s)

Backported it onto .19 because CI issues on Monday wound up delaying it until just after the branch happened, and it'd be nice to have the instrumentation this week rather than next week.

I've verified this event (feature: editCheck-paste, action: relevant-paste) is logging as expected in VisualEditorFeatureUse.

Since implemented on 18 September, it's been logged at least once within 13,576 distinct editing sessions across all Wikipedias. Events have been logged on both mobile and desktop and for both logged in and logged out users.

Note: Further analysis of the frequency of eligible Paste Check events will be completed as part of T403861