Since @cchen is out, I've requested the dbt-jobs repo rights for @kostajh to review/merge the adjustment to the metric: https://gitlab.wikimedia.org/repos/data-engineering/dbt-jobs/-/merge_requests/166

Thanks @BTullis!

I'm closing this since it's not possible to do accurately without a lot of work, which would take too much time.

In T433367#12192060, @Tchanders wrote:

Should we remove or disable the "Mark as false positive" button on handled revisions?

I'm thinking (1) it shouldn't be needed, and (2) it would provide a way to distinguish handled revisions when the Show handled revisions filter is checked.

Should we remove or disable the "Mark as false positive" button on handled revisions?

Is there any update on this task? My understanding is that permission is approved, but not granted yet.

The analysis has been run for eswiki, dewiki, frwiki and jawiki, and spreadsheets shared containing the top-scoring revision IDs for each wiki in the whole of June for labelling.

Recent edits pipeline + calling other classifiers added in this commit, and subsequent commits: https://gitlab.wikimedia.org/repos/product-safety-and-integrity/data-analysis/-/commit/41dbdbf83ede7caf00818213a36a81c6c8edcb32

The dataset and pipeline for generating it are at: https://gitlab.wikimedia.org/repos/product-safety-and-integrity/data-analysis/-/tree/main/projects/vandalism

Next step is to find some negative samples from the unreverted abuse filter hits. Looking through them, many seem like they should have been reverted (and probably were, but not in a way we can detect via tags or mediawiki_history) but some seem like genuine false positives.

I started with a first pass made from abuse filter hits, using this process:

• Choose a few filters from Special:AbuseFilter that seem to target silly vandalism
• Look up the time range when these filters were set to an action other than 'disallow' (e.g. 'warn') - because no edit is saved when they disallow
• Query for revision IDs of hits in that time range
• Check whether each one was reverted or not (using mediawiki_history.revision_is_identity_reverted)
• Positive samples are the reverted edits

Awaiting more information, to see if we can close this task - we may have captured everything essential in the topline metrics.

Date range: 27 March to 10 June 2026 (76 days)
Number of distinct users: 83
Number of wikis: 34

Updates are in a few merge requests, mostly:

• https://gitlab.wikimedia.org/repos/product-safety-and-integrity/data-analysis/-/merge_requests/12
• https://gitlab.wikimedia.org/repos/product-safety-and-integrity/data-analysis/-/merge_requests/14

Thanks @kevinbazira ! I think we will only need violation, p_violation and p_safe.

Pipeline is here: https://gitlab.wikimedia.org/repos/product-safety-and-integrity/data-analysis/-/tree/main/projects/suppressed_revisions

Let's leave this open until the backfills are complete. Dashboard is done though.

I don't think we need this any more, since we're pivoting away from the metric, and have analyzed the stability of it historically in T426598: Analyse stability of time to revert bad faith edit metric. We will keep tracking it going forwards, but I don't believe we need the historical comparisons on the dashboard.

This is done, and we are moving forward with the same-day handling metric: T428419: Set up Superset dashboard for monitoring DE4 metric same day handling of bad-faith content

In T417056#11987896, @Tchanders wrote:

Hi, I need to update my key again (hard disk failed, lost the old keys). Can we re-use this task?

Hi, I need to update my key again (hard disk failed, lost the old keys). Can we re-use this task?

There is still discussion over whether this metric needs any more changes, so let's leave this task open until those are resolved.

We will make a task for each metric, so I'm moving this back to In Refinement until that happens.

In T425292#11931394, @OKryva-WMF wrote:

I am thinking of "% none automatically identified accounts that were mitigated", i think this would not add additional complexity? If it's too hard to measure at this point we can de-prio this for now too.

Is there a way to measure manually identified accounts that were not mitigated?

In T425292#11929615, @OKryva-WMF wrote:

Thank you Thalia! I agree that some of these should be removed (updated the general description to remove them)

max_revert_seconds will be 7 * 86400 (7 days)