Page MenuHomePhabricator

Enable volunteers to explicitly configure how Tone Check behaves on a per project basis
Open, Needs TriagePublic

Description

Prompted by the 28 April Editing Team Community Call, and those that it spawned, this ticket involves the work of enabling volunteers to configure facets of how Tone Check behaves on a per project basis.

Story

Facets

This section will eventually contain the facets of Peacock Check's behavior that volunteers can explicitly control.

Configurable facetStatusNotes
Pages Peacock Check should never become activated on
Words/phrases/conventions Peacock Check should never consider to be non-neutral
Mode Tone Check is running inE.g. A) Silent: tag edits the model "thinks" non-neutral language is present within without presenting Tone Check to people while they're editing or B) Active: tag edits the model "thinks" non-neutral language is present within and present Tone Check to people while they're editing
Types of pages Peacock Check should never become activated on (e.g. biographies, political ideologies, active conflicts, etc.)✅ (T347775)
Confidence/probability threshold model would need to reach in order for Peacock Check to be presented to someoneSee mw:Edit_check/Configuration#Tone_check_configuration
People/account types Tone Check has the potential to activate forSee Edit_check/Configuration#Generic_check_configurationE.g. people acting in good faith (TBD how we might determine this), cumulative edit count, account state, etc.

Reference: T330112


Thank you to @SCP-2000 who prompted me to create this task.

Event Timeline

ppelberg updated the task description. (Show Details)
ppelberg moved this task from Untriaged to Upcoming on the Editing-team board.
ppelberg added a subscriber: Trizek-WMF.
ppelberg added a subscriber: SCP-2000.
ppelberg renamed this task from Enable volunteers to configure how Peacock Check behaves on a per project basis to Enable volunteers to explicitly configure how Peacock Check behaves on a per project basis.May 21 2025, 7:15 PM
Aklapper renamed this task from Enable volunteers to explicitly configure how Peacock Check behaves on a per project basis to Enable volunteers to explicitly configure how Tone Check behaves on a per project basis.May 28 2025, 11:43 AM
ppelberg updated the task description. (Show Details)
ppelberg updated the task description. (Show Details)

Quick technical speculation:

  • Pages: technically easy to add a pure blocklist of page-names to the config, but seems like a bad idea because it might quickly balloon out of control. Could otherwise define some signal that could be added to a page to block checks -- "if a page includes and of *these* templates", etc.
  • words / phrases / conventions: could blocklist specific exact strings that we would never even send to the model easily. If this is about more fine tuning of the model, which it feels "conventions", that's not really doable in a wiki-specific manner as I understand it.
  • Types: adding a "don't show on these specific categories" is trivial.
  • Confidence/probability threshold is implemented and just needs documentation now (T400210)
  • Mode: mostly a pain in that we'd need to exclude "the config says tone check shouldn't be shown in this specific way" from the existing "tag tone if the config would have allowed it" check, but doable.
  • People/account types: currently has edit count and login status. Could be expanded to other states -- roles, etc. "Good faith" is a challenge, though.
  • Page restrictions: feels like it might also fall into the "block if template X is on the page" camp.

Oh, "types" might also want to include namespaces: T400506

Next step
@ppelberg to decide what – if any – work is left to be done here.

Current config avil. here: https://www.mediawiki.org/wiki/Edit_check/Configuration

Next step
@ppelberg to decide what – if any – work is left to be done here.

Current config avil. here: https://www.mediawiki.org/wiki/Edit_check/Configuration

I've updated the task description to reflect the current status of Tone Check's configurability.

I'm moving this ticket to the backlog. If/when we see a need to prioritize work on the yet-to-be implemented facets, we'll prioritize work on this ticket.

If you'd like some sample text for this, then https://en.wikivoyage.org/wiki/Wikivoyage:Words_to_avoid has a list of words that should always be flagged. There will be false positives (e.g., "best" is acceptable for the "Best Western hotel chain" or "voted best chowder"), but these should not be common. There are translations into other pages, e.g., https://fr.wikivoyage.org/wiki/Aide:Mots_%C3%A0_%C3%A9viter and it's simpler and shorter than the English Wikipedia's https://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style/Words_to_watch