Page MenuHomePhabricator

Automoderator configuration: Revert threshold
Open, Needs TriagePublic

Description

User story: As an administrator, I want to set the caution level of Automoderator, so that its behaviour reflects my community's comfort level with false positives.

This ticket covers the technical implementation of this configuration and is not concerned with the user-facing interface, which may have different labels and options corresponding to those outlined here.


Name: Threshold
Options: Float values above 0.95
Default: 1


Administrators should be able to configure the threshold above which Automoderator will revert edits. Automoderator should revert any edit which equals or exceeds this value (while respecting the other rules set in T352439).

Any value below 0.95 should be treated as 0.95 - we don't want to allow communities to accidentally set the threshold far too low, resulting in a large number of false positive reverts.

Acceptance criteria

  • Automoderator only reverts edits which have a model score equal or exceeding the score set in the threshold configuration.
  • By default, the configuration is set to 1 (thereby not reverting any edits).
  • Scores below 0.95 are treated as if they are 0.95.

Event Timeline

We need to decide how thresholds/scores are defined (T360136) - this ticket may be incorrect in suggestion that the config would be a float directly.

We don't want wikis to have to maintain their scores based on Revert Risk model changes, if we control the thresholds we can adjust over time to maintain approx same accuracy etc.