Page MenuHomePhabricator

Build proof of concepts for Tone Check (mobile + desktop)
Closed, ResolvedPublic

Description

This task involves the work of implementing a proof of concept for the ML-backed Peacock Check (T379397) so that we can:

  • Encounter and confront UX questions/issues that still need to be addressed and ultimately, validate the usability of the experience via T383956
  • Evaluate what – if anything – about the Peacock detection model might need to be adjusted in order to enable the UX

Stories

  1. As an experienced volunteer/moderator motivated to ensure that Wikipedia offers people that is impartial, I want everyone contributing new text content in the main namespace to make an effort to ensure people encountering the information they are contributing will perceive it as neutral so that I am relieved of the burden of doing so myself.
  1. As someone who is new and motivated to contribute – what I consider to be – valuable + missing knowledge to Wikipedia, I want to know what I'm contributing adheres to relevant policies and conventions, so that I, and other people, can access this knowledge across time.

Requirements

The requirements that follow are drawn from Peacock Check/Requirements and Peacock, Feb 2025 (Figma)...
Meta

  • Platforms: Mobile + Desktop
  • Model: this proof of concept ought to be built in a way that it will be relatively straightforward to integrate the peacock language ML model when it's ready (late-March/early-April)

Detection (Mid-Edit)
Determine the presence/absence of peacock language...

  • Within < 500ms of someone finishing adding a new sentence/paragraph to a Wikipedia article (NS:0) that we are ≥90% sure contains peacock language present a Peacock Check card

Awareness + Understanding (Mid-Edit)
People are aware of what they have done/are doing that needs their attention and why...

  • The moment when the interface makes people aware that what they've written warrants additional attention will be the same on mobile and desktop; however, how the Peacock Check card is presented will vary by platform
    • On mobile, a visual hint will appear in the rail that exists alongside the editable surface. When tapped, the Peacock Check card will be expanded (T383955)
    • On desktop, the Peacock Check card will appear in the sidebar that exists alongside the editable surface. This card will contain:
      • Information that the software detected peacock/non-neutral language within the text you added
      • Information about why the presence of this language/tone is worth you allocating attention to.
        • E.g. Edits without non-neutral tone are ##% more likely to be approved (copy TBD).
      • Choices/calls to action to address the issue. See "Action" below.
  • The sentence(s) that the system has detected peacock language within ought to be highlighted and vertical yellow'ish bar will appear to further accentuate the text the system has detected Peacock language within

Action (Mid-Edit)
People are equipped with the tool(s)/affordance(s) to neutralize the tone of what they've written or explicitly decide not to and express why

  • Within the mobile and desktop Peacock Check cards, people will be met with the following choices:
    • A way to fix/address the tone issue the software has detected by tapping Revise. Upon tapping Revise:
      • Peoples' cursor ought to be placed at the very beginning of the text they added, and the system has detected Peacock Check language within. Note: on mobile keyboard should appear.
      • The text they originally wrote will remain highlighted, with the vertical yellow'ish bar still appearing beside it
    • A way to decline addressing the tone issue the software has detected by tapping Dismiss
      • Note: button label will likely change. We may also specify something else to show if/when people select this choice.
    • A way to collapse/defocus the Peacock Check card and bypass taking action on it during the "Mid-Edit" moment. Related: T386932.
    • A way to expand/refocus a Peacock Check card you had previously elected to collapse/defocus in the "Mid-Edit" moment. Related: T386932.
    • Any Peacock Check(s) that someone has not engaged with during the Mid-Edit moment should be re-presented during the Pre-Save moment with the same calls to action defined above
    • When someone revises a span of text that had caused the Peacock Check to activate, the model ought to re-check the text to evaluate the extent to which peacock/non-neutral language is still present.

Revisit (Pre-Save)
People are reminded to engage with Peacock Checks they did not address in the Mid-Edit moment.

  • Any Peacock Check that people did not explicitly engage with, by either tapping Revise or Dismiss, ought to be re-presented in the Pre-Save moment using Edit Check's "focus mode". Read: Checks shown one at a time, "stepper" to navigate between Checks, etc.
  • If someone taps Revise:
    • On mobile, they ought to be taken to an editable surface where:
        • Said text ought to be highlighted
      • Only the text the Peacock Check is relevant to is shown/available
      • The mobile VE editing tools are shown, flanked by < and > navigation buttons on either side
      • The > button ought to be deactivated until someone changes the text in someway
      • If someone where to tap < they ought to be taken back to the Pre-Save mode with the Peacock Check relevant to the text they were just viewing exposed
      • Once someone changes the text, the > will become activated and they'll be taken to one of two places:
        • If unaddressed Checks do NOT remain, they'll proceed to the edit summary
        • If unaddressed Checks do remain, they'll proceed back to the Pre-Save focus mode with any remaining Check(s) displayed
    • On desktop and mobile, they ought to be taken to the full editable surface
        • Upon arriving, they will see:
          • The text highlighted
        • The full VE editing toolbar will appear
        • The Publish Changes button ought to be deactivated until someone changes the text in someway
        • OPEN: will the Peacock Check "card" appear? What about the other unaddressed Checks?
      • Once someone changes the text, the Publish Changes button will become activated and they'll be taken to one of two places:
        • If unaddressed Checks do NOT remain, they'll proceed to the edit summary
        • If unaddressed Checks do remain, they'll proceed back to the Pre-Save focus mode with any remaining Check(s) displayed
      • OPEN: how might someone navigate back to the Pre-Save moment from here?

Related Objects

Event Timeline

I've updated the task description to include the choices we made offline this week.

Most notably, we decided that – to start – on desktop people who 1) do not engage with Peacock Check Mid-Edit and 2) elect Revise in the Peacock Check card that will be presented to them Pre-Save will be taken into the full VE surface to edit the text they'd written.

We made this choice with the following in mind:

  1. Creating a new, more-focused editing mode, involves non-trivial technical complexity
  2. Bringing people "back" to the editable surface could cause people to become confused overwhelmed

We felt comfortable risking "#2":

  1. Thinking it's important for us to try out the lightest weight approach to start and
  2. Knowing if/when we come to find this lightweight approach insufficient, we'll arrive back to the unmet UX goals with added conviction in the need to take on the technical complexity that could come with introducing a new mode within Edit Check

Update: 10 March
Per what @Esanders and I discussed offline today...

  • Mobile and desktop will share the same "Revisit (Pre-Save)" UX for now
    • Meaning, upon landing back into Mid-Edit from Pre-Save, the Mid-Edit experience will remain as it is currently. Read: text will be highlighted, on desktop, Peacock Card will be expanded, and other Checks have the potential of appearing if any are left unaddressed. We're going to defer the work of potentially excluding Checks that only show pre-save (e.g. Reference Check) for now...

I've updated the task description in T387901#10621481 to reflect the above.

Feedback: demo: 2e4d6c7a2d

  • Deactivate the "convert" checks that activate immediately upon arriving into the editor
  • Ensure that when you add peacock language to a paragraph that prompts the "puffery" dialog to appear, the paragraph is highlighted
    • At present, only the vertical bar appears
  • Ensure the "revise" button is functional in mid-edit
    • At present, tapping the "revise" button in the mid-edit moment appears to have no effect
  • Ensure this experience works well on mobile
    • E.g. integrates with the work @DLynch has been doing on the mobile "gutter"
  • Integrate the focusing/defocusing work @Zoë implemented

E.g. integrates with the work @DavidL has been doing on the mobile "gutter"

not me, but probably DLynch ?

E.g. integrates with the work @DavidL has been doing on the mobile "gutter"

not me, but probably DLynch ?

Shoot, yes! I'm sorry about that, @DavidL...

Update

Last week, @Esanders and I shared the first iteration of the PoC during design review.

While the feedback designers offered helpfully touched on all facets of the experience, the broader message we gleaned from this session was the following

There are enough rough edges within the experience to make it difficult for us to assess the viability of this approach to the Peacock Check experience.

With the above in mind, we're going to prioritize the adjustments listed below.

With these adjustments implemented, we'll then take another step back and invite staff and volunteers to review the experience.

Adjustments

Awareness + Understanding (Mid-Edit): people are aware of what they have done/are doing that needs their attention and why…

  • REVISE title of Check card to read "Review tone" instead of "Puffery"
  • 🚧 [not now] LIMT highlight to specific word(s)/phrase(s) that need attention
    • NOTE: T389434 needs to be resolved to determine the feasibility of doing this!!
  • 🚧 REVISE Check card copy to read as follows...
    • @ppelberg to specify copy that could cause people to think, "Oh, if I care about this edit being published, I should listen!" (and also include a way for people to "Learn more")

Action (Mid-Edit): people are equipped with the tool(s)/affordance(s) to neutralize the tone of what they've written or explicitly decide not to and express why

  • REVISE check dialog button copy from ReviseEdit and DismissKeep
  • REVISE what happens when you tap "Revise" mid-edit
    • Approach #1: Place cursor at the beginning of highlighted range, remove highlight, and replace with a border around the entire range (similar to what we do when focus is placed within nodes like links)
      • NOTE: @Esanders indicated that before we pursue this approach, we'd first need to think through how this would apply to other Checks
    • Approach #2: @Esanders can you think of an alternative to "#1" that could be effective at helping people A) quickly locate the cursor within the document and B) understanding the range of text that they're being asked to edit?
  • ADJUST sidebar so that the appearance of a Mid-Edit Check does not cause the page to reflow
    • Approach #1 Preserve enough space for Edit Checks cards – as currently designed – to appear without causing reflow
    • Approach #2: Offer a gutter-sidebar experience similar to what is shown on mobile and preserve enough space for an Edit Check to appear within it without the page needing to be reflowed/resized in anyway
      • We're going to start with this approach
    • Approach #3 Make Edit Check Cards/Hints dynamic and capable of adapting to the amount of white space available (might still cause reflow)
      • We might consider pursuing this approach as an extension to "Approach #2"
    • Approach #4: Do not show any sidebar at all (or only show a gutter-sidebar like on mobile), and instead cause check to appear through some other interaction (e.g., on hover, the sidebar pulls out over the edit surface)

Revisit (Pre-Save): people are reminded to engage with Peacock Checks they did not address in the Mid-Edit moment.

  • ENSURE that upon progressing to Pre-Save moment, sentence(s) with unaddressed Peacock Check issues are shown at the top of the viewport with the Check right alongside it
  • ENSURE article scrolls as you navigate between Checks

Address Check (Pre-Save → Mid-Edit): people electing to address a skipped Check can do so with focus and ease

  • REVISE Check card copy to acknowledge the unique moment (skip and then decide to address)
    • @ppelberg to propose copy that equips people with the guidance they will need to successfully "neutralize" the tone of what they've written
  • REVISE what happens when you tap Edit
    • See above
  • ADJUST sidebar so that the appearance of a Mid-Edit Check does not cause the page to reflow
    • Approach #2: Offer a gutter-sidebar experience similar to what is shown on mobile and preserve enough space for an Edit Check to appear within it without the page needing to be reflowed/resized in anyway

WIP from @DLynch: https://patchdemo.wmcloud.org/wikis/85aaed0085/w/index.php?title=Douglas_Adams&veaction=edit

Aklapper renamed this task from Build proof of concepts for Peacock Check (mobile + desktop) to Build proof of concepts for Tone Check (mobile + desktop).May 28 2025, 11:43 AM

A Tone Check PoC is available and has been available for testing via various patch demos for some months now. Tone Check is also available for testing on any Wikipedia now as well: https://www.mediawiki.org/wiki/Help:Edit_check#Testing_Tone_check