Page MenuHomePhabricator

Revise Tone: allow excluding article sections from suggestions
Open, Needs TriagePublic

Description

User story & summary:

As a newcomer trying Revise Tone for the first time, I want the suggestions I'm shown to be as accurate as possible, so that my edit is more likely considered productive and not reverted.

As a experienced editor or patroller reviewing edits, I want Revise Tone suggestions to be suppressed in sections where non-neutral language is intentional, so that I am not spending time reverting edits and feeling frustrated about Revise Tone suggested edits.

Background & research:

The Tone Check model currently evaluates article text with limited regard to which section that text sits in. Some sections are poor candidates for tone suggestions because non-neutral-sounding language in them is either expected, sourced to someone else, or not prose at all. Suggestions in those sections are noise: they cost newcomers time, and rejecting or acting on them can produce edits that experienced editors need to revert.

Potential sections that could be candidates to exclude:

  • Sections that consist of quoted or attributed opinion, where the non-neutral language belongs to the source rather than the article (e.g. "Reception", "Critical response", "Reviews", "Quote", "Quotation", "Text") (see also: T411892)
  • Plot summaries ("Plot", "Plot summary", "Synopsis", "Story", "Legend")
  • Sections that are lists or metadata rather than prose (e.g. "Discography", "Filmography", "Awards", "Bibliography")
  • Reference and appendix sections ("References", "Notes", "External links", "See also", "Further reading")
Proposed Solution

Add a per-wiki configurable list of section titles that Revise Tone skips when generating suggestions, or surfacing suggestions in the Suggested Edits feed.

Implementation is TBD, but the closest precedent is the "Add a Link" setting:

Screenshot 2026-09-10 at 11.18.01 AM.png (1,328×244 px, 53 KB)

Acceptance Criteria:

TBD: Needs discussion with engineers to determine approach and acceptance criteria.

Given [preconditions]
When [user action]
Then [outcome resulting from actions taken in the when clause]
All tasks should also meet the Growth Team’s task completion definition.

Event Timeline

Michael added subscribers: achou, Sucheta-Salgaonkar-WMF.

This is somewhere between "not possible" and "has to be done by Machine-Learning-Team, likely needing substantial effort and reworking their current approach". As I understand it, they are currently excluding sections in a language agnostic way based on the type of elements: References, Citations, Lists, Tables, Infoboxes, etc. They are no longer doing that with localized section names.

But this is not something the Growth Team can do, we're not creating the suggestions.

On a separate note, I'm not fully sure I understand why that story section should contain non-neutral language in wiki-voice. Someone else rewrote the section since the diff above, and to me the new version does read more appropriate: https://en.wikipedia.org/w/index.php?title=Etrian_Odyssey_Nexus&diff=1373898167&oldid=1373682336

Helping newcomers understand why their edit did not improve the situation is part of raising the next generation of editors. Maybe we can somehow make it easier for newcomers to learn that they are in fact allowed and expected to decline a suggestion if they do not know how to properly implement it? That is something Growth Team would be able to do.

If a specific article should really not be used in newcomer tasks, then Template:No_newcomer_task can already be used for that (as configured in CommunityConfiguration).

@Michael, could you say a bit more about why it wouldn't be possible to just adapt the existing functionality for excluding sections we have for Community Configuration of Add-Link-Structured-Task?

Story sections are allowed to deviate a little from standard NPOV. For instance, in the linked example, I wouldn't consider "gathered the realm's greatest explorers" to be non-neutral the same way I generally would if that language appeared elsewhere, because readers of the section understand that the section is describing the canon of the game rather than objective reality. And if the game stipulates that the explorers gathered are the greatest in the realm, then that's the canon. We do want to maintain a real-world rather than in-universe perspective for the article as a whole, but the "Plot" section title by itself establishes that section as the place to describe the fictional universe.

I don't see {{No newcomer task}} as a very good approach to this issue because it's a very blunt instrument. We'd have to apply it to all articles about fictional works, at which point we'd be missing non-neutral content in other sections.

On a separate note, I'm not fully sure I understand why that story section should contain non-neutral language in wiki-voice.

Because "neutral" is defined according to what the sources say, rather than according to whether the words fall in the middle of a normal curve. It's reasonable for the plot summary of a fairy tale like [[Cinderella]] to say something like "Dressed in the fine clothes her fairy godmother gave her, Cinderella was the most beautiful young woman at the ball", and it's also reasonable for the software to flag that as a sentence that would usually be non-neutral or unencyclopedic, i.e., it were presented as a claim of fact about a real human instead of merely being an accurate description of the basic points in the story.

Because "neutral" is defined according to what the sources say, rather than according to whether the words fall in the middle of a normal curve.

The onboarding quiz for Revise Tone attempts to teach this idea, but admittedly it's a complex idea to attempt to teach in a few sentences!

As I understand it, they are currently excluding sections in a language agnostic way based on the type of elements: References, Citations, Lists, Tables, Infoboxes, etc.

@achou would it be relatively easy to add to this list, or would it be impossible to add disallow suggestions in sections that are likely "plot summary" related?

Helping newcomers understand why their edit did not improve the situation is part of raising the next generation of editors.

I totally agree AND I can see why it's frustrating for patrollers to review these edits where it seems like the system made an incorrect suggestion.

@Michael, could you say a bit more about why it wouldn't be possible to just adapt the existing functionality for excluding sections we have for Community Configuration of Add-Link-Structured-Task?

This exclusion of sections has to be done as task generation. For Link Recommendations, this is done in the service that the Growth Team built back in the day and is owning since. But for Revise Tone, we are not doing the task generation, that is done by an Airflow Pipeline built by the Machine Learning Team, and they would then have to somehow integrate that language-specific excluded sections information into their pipeline (which works on a per-edit basis).
I'm not familiar with their technology stack, but my guess would be that it requires some substantial amount of complexity to be added to make this work with the current setup. And it will likely need some manual work whenever that list of sections is expanded, because that would mean that the stored tasks need to be regenerated.
This is (probably) possible, but definitely a product call affecting multiple teams.

This might become much simpler if/when we move to the new way of showing tasks from various sources on the new Home page, because then we might be sourcing these tasks in a different way (using the Editing Team's Edit Suggestions provided by the headless VE, if that or an alternative works out). But that is still well into the future and it is not yet decided if we're even going to go that direction.

@achou would it be relatively easy to add to this list, or would it be impossible to add disallow suggestions in sections that are likely "plot summary" related?

This should be doable, but it's not a small change if we want to do it in a maintainable way.

There are a few ways we could do it. One option is for the ML generation pipeline to keep the per-wiki excluded section list and apply the filtering there. That would work, but every time the list changes, we'd need to update it and redeploy the generation pipeline for the change to take effect.

There might be other options too. For example, if the section name were stored with the generated suggestion in cassandra, maybe Growth could do the filtering instead. Or if this kind of section information could somehow come through the page_change events that the generation pipeline already uses, that could also make things easier.

Those options would need some input from Data Persistence and Data Platform teams, so I think it's worth discussing with them before we decide where this should live.

So overall: it's possible, but how much work it is depends quite a bit on which approach we take.

Thanks for the explanation, Aiko!
Since this isn't a small change, it might make sense to first focus on other Add a Link improvements:
Growth just started T429417: Add Link should not make a recommendation if one has already been accepted or declined (Limit an article to only one Add a Link suggestion), and Machine Learning just finished T434259: Introduce case sensitivity to Add-a-Link and is now working on T435831: Incorporate user decision signals into Add Link model training.

My hope is that once all of those changes land we will see significant improvements in suggestion accuracy. And at that point we can decide if this task is still necessary and worth the investment or if we want to consider alternate approaches (improved onboarding, etc.).