Paste Check: Instrument the Paste Check UX
Closed, ResolvedPublic
Actions

Description

This ticket involves the work of implementing the instrumentation required to track how people are engaging with the Paste Check UX.

The instrumentation required to do the above was largely added as part of the Multi-Check feature implementation (T352092).

As such, this ticket largely holds the work of verifying the ===Requirements have been implemented as expected.

Requirements

Note: These are still draft pending finalization of the UX workflow

feature	action	meaning
editCheck-paste	action-keep	User clicks 'Yes, keep it' button on Paste Check
editCheck-paste	action-remove	User clicks 'No, remove it' button on Paste Check
editCheck-paste	check-shown-presave; check-shown-midedit	Paste check was shown pre-save or mid-edit. An event will be sent for every check shown.
editCheck-paste	edit-check-feedback-shown	Decline survey is presented after user elects to keep text
editCheck-paste	edit-check-feedback-reason-[whatever you picked]	User selects a decline survey option and clicks 'submit'
editCheck-paste	relevant-paste	Logs when Paste Check would have been shown to someone in the control group
editCheck-paste	click-learn-more	User clicks on the learn more about copyright violation link within Paste Check

instrumentation spec

Progress

Next steps

Megan to finish populating requirements
David to confirm instrumentation is implemented
Esther/Rummana to QA
Megan to QA

Done

Editing QA verifies ===Requirements are being logged client-side
@MNeisler verifies ===Requirements are being logged server-side

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	Edit check: change the name of the paste check from importCopyvio	mediawiki/extensions/VisualEditor	master	+1 -1

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Open	None	T265163 Create a system to encode best practices into editing experiences
In Progress	nayoub	T359107 Paste Check: Prompt people pasting text to consider risk of copyright violation
Resolved	Ryasmeen	T405127 [MILESTONE] Deploy Paste Check to all Wikipedias
Open	ppelberg	T392954 [FY25-26] WE 1.1: Increase constructive edits
Resolved	MNeisler	T399669 [WE 1.1.1] Run an A/B test to evaluate impact of Paste Check
Resolved	ppelberg	T400096 Paste Check: Instrument the Paste Check UX

Event Timeline

ppelberg created this task.Jul 21 2025, 7:04 PM

ppelberg moved this task from Backlog to Analytics on the Editing-team (Tracking) board.

MNeisler claimed this task.Jul 29 2025, 6:27 PM

MNeisler added a project: Product-Analytics (Kanban).

MNeisler moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.Jul 31 2025, 6:08 PM

MNeisler triaged this task as Medium priority.Jul 31 2025, 6:49 PM

MNeisler updated the task description. (Show Details)Aug 1 2025, 6:13 PM

MNeisler updated the task description. (Show Details)Aug 1 2025, 6:16 PM

MNeisler updated the task description. (Show Details)Aug 1 2025, 6:18 PM

MNeisler updated the task description. (Show Details)

Currently the feature name is going to be editcheck-importCopyvio. I'm not averse to changing it, if desired.

ppelberg edited projects, added Editing-team (Kanban Board); removed Product-Analytics (Kanban), Editing-team (Tracking).Aug 6 2025, 4:16 PM

ppelberg edited projects, added Editing-team (Planning); removed Editing-team (Kanban Board).

ppelberg moved this task from Inbox to Q3 2025-2026 on the Editing-team (Planning) board.Aug 6 2025, 4:21 PM

In T400096#11054837, @DLynch wrote:

Currently the feature name is going to be editcheck-importCopyvio. I'm not averse to changing it, if desired.

Good spot. Can we please change the name of the feature to reflect the name of the Check (Paste Check)?

Change #1176341 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/VisualEditor@master] Edit check: change the name of the paste check from importCopyvio

https://gerrit.wikimedia.org/r/1176341

gerritbot added a project: Patch-For-Review.Aug 7 2025, 4:02 AM

Change #1176341 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Edit check: change the name of the paste check from importCopyvio

https://gerrit.wikimedia.org/r/1176341

ReleaseTaggerBot added a project: MW-1.45-notes (1.45.0-wmf.14; 2025-08-12).Aug 7 2025, 12:00 PM

Maintenance_bot removed a project: Patch-For-Review.Aug 7 2025, 12:30 PM

@DLynch: would it be accurate for us to think the events Megan specified in the task description are implemented?

I ask the above seeing https://gerrit.wikimedia.org/r/1176341 and assuming, perhaps inaccurately, that that match is scoped specifically to renaming the feature.

Apart from the last two. There's no check-control-[listener] because we haven't implemented any a/b test code for this yet. There's no check-learn-more because there's not a learn more link in the check yet.

Regarding learn-more: we last did some copy updates when we refreshed the flow in T400518, at which time we hadn't specified a link, and then said we weren't going to keep fiddling with it until T400518 dealt with it.

ppelberg moved this task from Inbox to Doing on the Editing-team (Kanban Board) board.Aug 18 2025, 4:45 PM

Pikne unsubscribed.Sep 10 2025, 7:00 AM

In T400096#11088193, @DLynch wrote:

Regarding learn-more: we last did some copy updates when we refreshed the flow in T400518, at which time we hadn't specified a link, and then said we weren't going to keep fiddling with it until T400518 dealt with it.

@DLynch, what do you think the next step here is?

I ask this knowing we've sorted out the outstanding copy questions in T400518.

ppelberg reassigned this task from MNeisler to DLynch.Sep 23 2025, 5:11 PM

I can confirm that Ed implemented the click-learn-more event in the patch that adjusted the copy to include the link.

We did wind up not having the check-control- event, because T402460 asked for something that happened regardless of the a/b test. It'll be equivalent to looking for`relevant-paste` events with a control bucket.

In T400096#11210971, @DLynch wrote:

We did wind up not having the check-control- event, because T402460 asked for something that happened regardless of the a/b test. It'll be equivalent to looking for`relevant-paste` events with a control bucket.

Per offline discussion with @MNeisler, David what you described above sounds great.

ppelberg moved this task from Doing to QA on the Editing-team (Kanban Board) board.Sep 24 2025, 8:34 PM

ppelberg added a project: Editing QA.

MNeisler updated the task description. (Show Details)Sep 24 2025, 8:46 PM

ppelberg moved this task from Inbox to High Priority on the Editing QA board.Sep 24 2025, 10:22 PM

ppelberg updated the task description. (Show Details)

JJMC89 unsubscribed.Sep 24 2025, 10:25 PM

Ryasmeen edited projects, added Verified; removed Editing QA.Sep 26 2025, 8:25 PM

ppelberg assigned this task to MNeisler.Sep 26 2025, 10:43 PM

ppelberg edited projects, added Editing-team (Tracking); removed Editing-team (Kanban Board).

ppelberg updated the task description. (Show Details)

MNeisler added a project: Product-Analytics (Kanban).Oct 2 2025, 6:25 PM

I've confirmed that all paste check events identified in the task description are logging as expected in VisualEditorFeatureUse.

This is based on a review of test events generated on various wikis since events were instrumented. Note: This excludes a review of the Paste Check eligible event (relevant-paste), which was QA'd separately in T402460.

See summary of the checks completed below:

Paste Check events have been logged at 24 editing sessions by 11 distinct users since we first started recording events on 23 August 2025.
Paste Check events have been logged at cswiki, enwiki, frwiki, and ptwiki to date.
Paste Check has been shown only once within an editing session and multiple times within an editing session as expected.
All expected paste check events have been logged except action = check-shown-presave, which appears expected based on this thread
Paste Check events are logged for both desktop and mobile platforms
Paste Check events are logged for both unregistered and registered users.
The order of events within a session appears as expected (i.e. check-shown-midedit appears before action-keep or action-remove)
All events are recorded while the user is editing with VisualEditor (editor_interface = 'visualeditor')

The VEFU data dictionary has been updated to include these events.

cc @ppelberg

MNeisler reassigned this task from MNeisler to ppelberg.Oct 6 2025, 3:52 PM

MNeisler moved this task from Doing to Needs Review on the Product-Analytics (Kanban) board.

All expected paste check events have been logged except action = check-shown-presave, which appears expected based on this thread

We can consider this task complete. Although, Peter needs to follow up with David L. to learn more about why Paste Checks are not being shown pre-save.

Paste Check: Instrument the Paste Check UXClosed, ResolvedPublicActions