Page MenuHomePhabricator

enable admins to enable and configure the multilingual revert risk model via Community Config
Closed, ResolvedPublic5 Estimated Story Points

Description

Details taken directly from T380367: How should we enable administrators to select and configure the multilingual revert risk model on Automoderator's configuration page?:
We would like to move to using the multilingual revert risk model in place of the language-agnostic model on the wikis it supports, as it has a higher accuracy overall. However, per the research done on this model, it has a much more varied performance depending on the specific language it is being used on

Designs

Figma link

Acceptance Criteria:

  • Updated copy to the text below the "Configure threshold" title.
  • 2 more headers 1) Language-agnostic model 2) Multilingual model
  • Below each header, there is a checkbox with a "[model] is turned on" option.
  • When one of the models is turned on (by checking one of the checkboxes), the checkbox for the other model should be disabled.
  • Additional copy below each checkbox that gives details about each model.
  • For the multilingual model, admins can input a threshold between 0.950 and 0.999.
  • If the value that is inputted falls outside of the range, an inline error message is displayed: "The value is outside the range. Please input a value that is between 0.950 and 0.999"
  • The multilingual option should only be displayed in communities which the multilingual model supports. For other wikis, we should only display the current enabled/disabled option.
Design+Error message
Multilingual model.png (1,440×2,505 px, 272 KB)
Multilingual model_ Error message.png (1,440×2,505 px, 281 KB)

Event Timeline

There is one unresolved issue with these designs - we want to only show multilingual as an option on the wikis it supports, which, as I understand it, is possible. The problem is that the text we have in the mockups assumes that both are visible. Is it possible to change the help text for the section and other config, based on multilingual support? Or is there a way we could rewrite the text so that it works whether one or two options are shown?

[...]
Is it possible to change the help text for the section and other config, based on multilingual support?
[...]

I believe so. We can use different configuration providers based on the wiki id.

Kgraessle set the point value for this task to 5.Mar 4 2025, 4:14 PM
Kgraessle moved this task from To be estimated to Estimated on the Moderator-Tools-Team board.
DMburugu triaged this task as Medium priority.Mar 4 2025, 5:14 PM

Change #1131495 had a related patch set uploaded (by Scardenasmolinar; author: Scardenasmolinar):

[mediawiki/extensions/AutoModerator@master] Add new Community Config form for multilingual config

https://gerrit.wikimedia.org/r/1131495

Change #1131495 merged by jenkins-bot:

[mediawiki/extensions/AutoModerator@master] Add new Community Config form for multilingual config

https://gerrit.wikimedia.org/r/1131495

For the multilingual model, admins can input a threshold between 0.950 and 0.999.

This might have actually been too high, given the analysis we did in T385102. Matching the estimated error range of the language agnostic model when set at 0.975 gives a multilingual score range starting at 0.87. I'll file a follow-up task.