Page MenuHomePhabricator

Local revert-risk scoring and a per-factor explain page for AutoModerator
Open, Needs TriagePublic

Description

Proposal

Two related additions to AutoModerator built together as a prototype and proposed for discussion. We may not deploy either to production.

Local scoring.
AutoModerator scores every edit by calling the LiftWing revert-risk endpoint, or by reading an ORES score. That external dependency makes it harder to deploy. A deterministic local backend reproduces the language-agnostic model in pure PHP: the same score, no network call, no new database storage. It is selectable alongside the existing language-agnostic and multilingual backends. This is only available for the language agnostic model because the XGBoost algorithm can be implemented in PHP, but the multilingual model is a BERT transformer.

Explain page.
Even with a score in hand, there is no on-wiki way to see why a given edit scored the way it did. A Special:AutoModerator page takes a revision ID and shows a per-factor breakdown: the overall probability of revert, the model's base rate, and how each signal (editor experience, page age, wikilinks, references, etc) moved the score up or down, ranked by how much it mattered.

The two connect: scoring locally means walking the model's trees in PHP, which is what makes the per-factor Shapley attribution possible. Each also stands on its own; local scoring is useful without the explain page, and the explain page works against any wiki running the local backend.

There is a working prototype. Implementation notes and references are in the first comment.

Screenshots (explain page)

Desktop overview:

explain-desktop-collapsed.png (1,539×2,188 px, 340 KB)

A factor expanded, showing the underlying feature values and the score caveat:

explain-desktop-expanded.png (1,539×2,833 px, 541 KB)

Mobile:

explain-mobile.png (758×2,539 px, 298 KB)

Open questions

  • Audience, and where the explain link appears: the automoderator revert log entry, the reverted user's talk-page message, or just leave it buried?
  • The explain page re-scores on demand, so live inputs (the editor's current edit count and rights, the page's current history) drift from the score AutoModerator acted on when it made the revert. Good enough, or do we persist the score at revert time? I considered dumping the SHAP output into a hidden log entry so that the explanation page could look up the historical values.
  • How does the committed model artifact get refreshed when the upstream model changes? and how we gate that refresh on fidelity to LiftWing?
  • Language-agnostic-only. Wikis running the multilingual model get neither the local score nor the explanation. Is this useful enough as a "language agnostic only" feature?

Acceptance criteria

Local scoring

  • Scores a main-namespace revision locally, reproducing the language-agnostic model within epsilon of LiftWing, with no LiftWing or ORES call.
  • Selectable via site config alongside the language-agnostic and multilingual backends, reusing the existing caution-level threshold.
  • No new database storage on the scoring path.

Explain page

  • Given a main-namespace revision ID, the page shows the revert probability, the base rate, and a per-factor contribution breakdown.
  • Each factor shows its contribution, its direction (raises or lowers risk), its share of the total movement, and the underlying feature values.
  • The page renders on desktop and mobile.
  • A disclaimer states the score can be wrong and that it is recomputed on demand.

Event Timeline

How the scoring works

  • The local scorer reproduces the language-agnostic revert-risk model in pure PHP: it walks the model's XGBoost trees, sums leaf margins, and applies the sigmoid, matching LiftWing to within epsilon.
  • Features come from the raw wikitext of the revision and its parent (link, category, heading, media, and reference counts; text length in code points), plus account and page signals (editor age, edit count, user rights, page age).
  • It is selected by site config alongside the language-agnostic and multilingual backends and reuses the existing caution-level threshold, since the local score sits on the same probability scale as LiftWing.
  • The committed model artifact is refreshed by an engineer-run offline converter; a fidelity test against LiftWing is the gate on that refresh.
  • Per-factor contributions on the explain page are TreeSHAP (Shapley) values over that same tree walk, so the factor shares sum back to the movement from the base rate.
  • No LiftWing or ORES call and no new database storage; the score and its breakdown are computed on demand.

References

Change #1131416 had a related patch set uploaded (by Jsn.sherman; author: Jsn.sherman):

[mediawiki/extensions/AutoModerator@master] Scoring: Add a local revert-risk backend

https://gerrit.wikimedia.org/r/1131416

Change #1319849 had a related patch set uploaded (by Jsn.sherman; author: Jsn.sherman):

[mediawiki/extensions/AutoModerator@master] Scoring: Extract shared loader and feature vector

https://gerrit.wikimedia.org/r/1319849

Change #1319850 had a related patch set uploaded (by Jsn.sherman; author: Jsn.sherman):

[mediawiki/extensions/AutoModerator@master] Scoring: Add TreeSHAP attribution by concept

https://gerrit.wikimedia.org/r/1319850

Change #1319851 had a related patch set uploaded (by Jsn.sherman; author: Jsn.sherman):

[mediawiki/extensions/AutoModerator@master] Scoring: Explain a revision on CLI and web

https://gerrit.wikimedia.org/r/1319851

Change #1319852 had a related patch set uploaded (by Jsn.sherman; author: Jsn.sherman):

[mediawiki/extensions/AutoModerator@master] Scoring: Run local inference in the revert job

https://gerrit.wikimedia.org/r/1319852

Change #1319892 had a related patch set uploaded (by Jsn.sherman; author: Jsn.sherman):

[mediawiki/extensions/AutoModerator@master] Scoring: Per-factor readout on the explain page

https://gerrit.wikimedia.org/r/1319892