Page MenuHomePhabricator

Automoderator configuration: Revert threshold
Closed, ResolvedPublic3 Estimated Story Points

Description

User story: As an administrator, I want to set the caution level of Automoderator, so that its behaviour reflects my community's comfort level with false positives.

This ticket covers the technical implementation of this configuration and is not concerned with the user-facing interface, which may have different labels and options corresponding to those outlined here.


Name: Threshold
Options: Float values above 0.95
Default: 1


Administrators should be able to configure the threshold above which Automoderator will revert edits. Automoderator should revert any edit which equals or exceeds this value (while respecting the other rules set in T352439).

Any value below 0.95 should be treated as 0.95 - we don't want to allow communities to accidentally set the threshold far too low, resulting in a large number of false positive reverts.

Acceptance criteria

  • Automoderator only reverts edits which have a model score equal or exceeding the score set in the threshold configuration.
  • Threshold settings below 0.95 are treated as if they are 0.95.

Event Timeline

We need to decide how thresholds/scores are defined (T360136) - this ticket may be incorrect in suggestion that the config would be a float directly.

We don't want wikis to have to maintain their scores based on Revert Risk model changes, if we control the thresholds we can adjust over time to maintain approx same accuracy etc.

Part of this is implemented in T352675: Automoderator Mediawiki configuration: Edit summary. What isn't is the guardrail for 0.95 being the minimum allowable value, even in MediaWiki configuration. That guard clause is outstanding work.

jsn.sherman set the point value for this task to 3.Apr 16 2024, 3:24 PM
jsn.sherman moved this task from To be estimated to Up next on the Moderator-Tools-Team board.

Change #1024469 had a related patch set uploaded (by Kgraessle; author: Kgraessle):

[mediawiki/extensions/AutoModerator@master] Automoderator configuration: Revert threshold

https://gerrit.wikimedia.org/r/1024469

Change #1024469 merged by jenkins-bot:

[mediawiki/extensions/AutoModerator@master] Automoderator configuration: Revert threshold

https://gerrit.wikimedia.org/r/1024469