Create Tone Check Patch demo for concept validation
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	ppelberg
	Apr 29 2025, 4:21 AM

Description

This task involves the work of building a Patch demo prototype that the Design Research Team (specifically, @MRaishWMF) can use during the course of the interviews we will be conducting in T391162.

Requirements

Prioritized highest --> lowest

Implement a probability score threshold so that a Peacock Check is shown only when the probability score is ≥0.8 or higher.
Implement new copy, per T390248
Suppress the dialog that prompts people to choose between the Real API and "amazing" Fake
Fix issue where Peacock Check is shown if you simply add a space to a new paragraph
- See screen recording of issue.
Ensure the Peacock Check card is displayed during the Pre-Save moment if someone does not elect to Revise or Decline during the Mid-Edit moment.
- At present, if someone skips the Peacock Check during the Mid-Edit moment, the Check is not resurfaced during Pre-Save.

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	Make peacock check more demo-ready	mediawiki/extensions/VisualEditor	master	+10 -23

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Open	None	T265163 Create a system to encode best practices into editing experiences
Open	None	T365300 Introduce Edit Checks that encourage behavior aligned with Manual of Style consensus
Open	None	T365301 Tone Check: Prompt people to revise promotional language
Open	ppelberg	T392954 [FY25-26] WE 1.1: Increase constructive edits
Open	Quiddity	T409513 Publish Tone Check leading indicator findings on mediawiki.org
Open	MNeisler	T387918 [MILESTONE] Run an A/B test to evaluate impact of Tone Check
Resolved	nayoub	T383956 [MILESTONE] Conduct usability tests for Tone Check proof of concept
Resolved	MRaishWMF	T391162 [FY25-WE.1.2.13] Tone Check concept validation
Resolved	zoe	T392871 Create Tone Check Patch demo for concept validation

Event Timeline

ppelberg created this task.Apr 29 2025, 4:21 AM

ppelberg moved this task from Inbox to Ready to Be Worked On on the Editing-team (Kanban Board) board.

Excited for this! Can we add a requirement about the demo implementing a probability score threshold (eg. only surfacing the peacock check when the probability score is 0.8 or higher)? This will allow us to demo something that is closer to what we expect the production experience to look like. We could also test different thresholds during these interviews if it makes sense to do so.

In T392871#10776491, @SSalgaonkar-WMF wrote:

Excited for this! Can we add a requirement about the demo implementing a probability score threshold (eg. only surfacing the peacock check when the probability score is 0.8 or higher)? This will allow us to demo something that is closer to what we expect the production experience to look like. We could also test different thresholds during these interviews if it makes sense to do so.

Oh, yes! Great spot, @SSalgaonkar-WMF. Let's do as you proposed: implement a probability score threshold so that a Peacock Check is shown only when the probability score is ≥0.8 or higher.

I've updated the task description to reflect this.

ppelberg updated the task description. (Show Details)Apr 29 2025, 3:47 PM

Change #1140499 had a related patch set uploaded (by Zoe; author: Zoe):

[mediawiki/extensions/VisualEditor@master] Make peacock check more demo-ready

https://gerrit.wikimedia.org/r/1140499

gerritbot added a project: Patch-For-Review.May 1 2025, 1:29 PM

Test wiki created on Patch demo by Zoe-WMF using patch(es) linked to this task:
https://patchdemo.wmcloud.org/wikis/d2d13f14a1/w/

Probability threshold implemented and set to 0.8
Implemented new copy
- Note: Not for mid-edit spawned from pre-save – still looking in how to detect that case
Suppressed the dialog, switched to using the real API
Peacock check no longer shows on every change due to the 0.8 threshold

In progress:

Ensuring pre-save shows if mid-edit was ignored
Different messaging if the user came from pre-save

Fired up a patchdemo:

Screenshot 2025-05-01 at 14.44.35.png (582×1 px, 100 KB)

Test wiki created on Patch demo by Zoe-WMF using patch(es) linked to this task:
https://patchdemo.wmcloud.org/wikis/ef38198cb5/wiki/Lemon

ppelberg moved this task from Ready to Be Worked On to Doing on the Editing-team (Kanban Board) board.May 1 2025, 6:23 PM

(removed the Research tag given that Zoe is working on it and this is not a request for Research to triage. Thanks.)

Next steps

@ppelberg to review patch demo
@zoe to file new ticket for requirements for "5."

In T392871#10797164, @ppelberg wrote:

Next steps

@ppelberg to review patch demo

@zoe, https://patchdemo.wmcloud.org/wikis/d2d13f14a1/wiki/Main_Page is coming along nicely.

Beyond the "in progress" work [i] that will happen in a new ticket, I see one thing remaining to close this ticket out...

Could you pelaseto update the button copy within the Peacock Check card so that it reads Decline instead of Ignore? [ii]

Screenshot 2025-05-06 at 13.35.14.png (434×790 px, 37 KB)

i. In progress via T392871#10783331:

Ensuring pre-save shows if mid-edit was ignored
Different messaging if the user came from pre-save

ii. Note: the button copy might be fixed in the latest patch demo you seemed to have created (ef38198cb5). Although, I've not yet been able to get the Check to activate and subsequently, review the button copy.

ppelberg assigned this task to zoe.May 6 2025, 8:42 PM

zoe mentioned this in T393570: Ensure pre-save edit checks know whether a user has interacted with mid-edit equivalents.May 7 2025, 10:27 AM

zoe added a subtask: T393570: Ensure pre-save edit checks know whether a user has interacted with mid-edit equivalents.

I've updated the copy (without translation infrastructure) however I'm unable to create a patchdemo at present due to an error outside of my control – I'll try again in a little bit.

VPuffetMichel added a project: OKR-Work.May 8 2025, 1:54 PM

In T392871#10800295, @zoe wrote:

I've updated the copy (without translation infrastructure) however I'm unable to create a patchdemo at present due to an error outside of my control – I'll try again in a little bit.

Sounds great, @zoe. Let's track the change to revise Ignore to Decline in T390248.

And with work on requirement "5." now happening in T393570, let's consider this task Resolved.

Test wiki on Patch demo by Zoe-WMF using patch(es) linked to this task was deleted:

~~https://patchdemo.wmcloud.org/wikis/d2d13f14a1/w/~~

Test wiki on Patch demo by Zoe-WMF using patch(es) linked to this task was deleted:

~~https://patchdemo.wmcloud.org/wikis/ef38198cb5/w/~~

ppelberg removed a subtask: T393570: Ensure pre-save edit checks know whether a user has interacted with mid-edit equivalents.May 20 2025, 5:15 PM

Change #1140499 abandoned by Zoe:

[mediawiki/extensions/VisualEditor@master] Make peacock check more demo-ready

Reason:

See I9d6efdfce20bed162ea20e6268bd14af6290c063

https://gerrit.wikimedia.org/r/1140499

Maintenance_bot removed a project: Patch-For-Review.May 23 2025, 1:30 PM

Aklapper renamed this task from Create Peacock Check Patch demo for concept validation to Create Tone Check Patch demo for concept validation.May 28 2025, 11:43 AM

	F59725183: Screenshot 2025-05-06 at 13.35.14.png
	May 6 2025, 8:42 PM

	F59605416: Screenshot 2025-05-01 at 14.44.35.png
	May 1 2025, 1:47 PM

Create Tone Check Patch demo for concept validationClosed, ResolvedPublicActions