Definition:

• The median time-to-live of bad-faith content that is mitigated within 7 days on each of the top 5 Wikipedias. This means the median time between when a bad-faith edit was attempted (meaning a save attempt was made), and when a mitigating action was taken against it.
• Where these top 5 Wikipedias are: English, German, Spanish, Japanese, and French.
• Where bad-faith activity is defined based on who reverts the edit - tracking therefore edits reverted by UWERs (checkuser, suppress, sysop, rollbacker, abusefilter, abusefilter-helper). This is a signal for abusive activity (e.g., threats, doxxing, vandalism, and disruption) and inauthentic activity (e.g., undisclosed paid editing, bots, campaigns to reduce knowledge integrity).
• Where 7 days means we exclude edits that were reverted more than 7 days after the edit was made
• Where mitigating action is taken either directly by a UWER or by an automated decision taken by a centrally-managed system component whose focus is bad-faith content
• Measurement plan: https://docs.google.com/document/d/1A5CaMmReO12a6toLde06OSLex-F1DdrILQV_uu6K-CY/edit?tab=t.0

Data Source:

• Does the data exist? Yes - <should I add the table here or link doc? Tiger team to advise?>
• If no - what is the plan / alternative proposal?

Baseline:

• enwiki: 55m24s
• dewiki: 31m27s
• eswiki: 41m20s
• jawiki: 42m30s
• frwiki: 36m24s

Add link.

Target:

• 50% reduction in this median time within a 7 day period; separately measured per wiki

Error (if applicable):

• TBD

Automated Dashboard:

• Is there already an automated dashboard for it? Confirming if yes we'll add the link here.

Test Kitchen Status:

• TBD

@EMill-WMF confirm rationale for 7 day window:

There is a very long tail involving edits that took a long time to revert. Two reasons to exclude these edits:

• This long tail makes the median/means look a lot bigger, even if most reverts were done within minutes
• We expect that these represent edits that are more complicated to recognize as something that should be reverted, and we are focusing initially on the larger volume, easier to detect bad edits

Also @EMill-WMF confirm reason for choosing those top 5 wikis please.

Hello all- While I can see that the post at https://phabricator.wikimedia.org/T247181 is from me, I don't remember what prompted my posting. I don't know about choosing the top 5 wikis, not sure what that's about, sorry.

I think the wrong person was subscribed here; removing - sorry for the noise.

ARCHIVED for visibility (Prior definition, baseline, target and plan):

Definition:

The median time-to-live of bad-faith content that is mitigated within 7 days on each of the top 5 Wikipedias. This means the median time between when a bad-faith edit was attempted (meaning a save attempt was made), and when a mitigating action was taken against it.
Where these top 5 Wikipedias are: English, German, Spanish, Japanese, and French.
Where bad-faith activity is defined based on who reverts the edit - tracking therefore edits reverted by UWERs (checkuser, suppress, sysop, rollbacker, abusefilter, abusefilter-helper). This is a signal for abusive activity (e.g., threats, doxxing, vandalism, and disruption) and inauthentic activity (e.g., undisclosed paid editing, bots, campaigns to reduce knowledge integrity).
Where 7 days means we exclude edits that were reverted more than 7 days after the edit was made
Where we use the identity revert definition because of the convenient revision_seconds_to_identity_revert field
Where mitigating action is taken either directly by a UWER or by an automated decision taken by a centrally-managed system component whose focus is bad-faith content

Measurement plan:

https://docs.google.com/document/d/1A5CaMmReO12a6toLde06OSLex-F1DdrILQV_uu6K-CY/edit?tab=t.0

Data Source:

Does the data exist?
Yes - for monthly compute. wmf.mediawiki_history
No - for weekly compute. Implementation of T424350: Incremental MediaWiki History v1 - Phase I is needed. Target delivery is end of May.

Baseline:

The baseline is the median time to revert over the 180-period from 26 October 2025 to 24 April 2026:
enwiki: 40m22s
dewiki: 10m49s
eswiki: 25m24s
jawiki: 39m49s
frwiki: 37m35s
Link to baseline calculations: https://gitlab.wikimedia.org/repos/data-engineering/dbt-jobs/-/blob/main/models/product-safety-and-integrity/marts/time_to_revert_bad_faith_edits.sql
Query in P91457
Additional note about baseline: Only the manual reverts are included in our baseline, because we are only targeting improvement of these. There are automated actions that already take place - e.g. AbuseFilter preventing save, community bot automatically reverting. These are not included in our baseline because we are not targeting these
Baseline update: we will compare this against the full-year data once backfilled to verify if there’s any significant annual seasonality we should account for.

Target:

50% reduction in this median time within a 7 day period; separately measured per wiki

Error (if applicable):

TBD

Automated Dashboard:

https://superset.wikimedia.org/superset/dashboard/788

This is awaiting some backfills: T425526

The metric will be measured monthly until the weekly history tables are available, and then it will be weekly. Each measurement will be the median over the previous 180 days. 180 days was needed because smaller periods of time are too noisy.

Test Kitchen Status:

TBD

Additional context:

• 7 day window: We chose a window mainly to exclude large outliers from skewing the results. And, because we do expect the biggest value of the work to come from making the day-to-day tussle of anti-abuse work more efficient.

@Tchanders is working through the dbt query adjustments. DE will provide backfilling once dbt model is finalized.

Data clean up and backfilling in progress, cc @amastilovic
https://superset.wikimedia.org/superset/dashboard/788/?native_filters_key=m232aBVMTER56HRV4Y2FQBJYJhkQdWWcjFAIEE2Jtj37omYd2x9pKyVN0ixHzzy6

Status Update:

• Definition needs: Incremental MW, Approval
• Implementation needs: Test Kitchen Sign Off, Approval
• Active Data Issue: Yes (metric has become noisier, active discussion on de4-metrics-tiger-team channel and https://phabricator.wikimedia.org/T434812)
  • Status:
    • It was noted the metric showed unusual noise. After considering factors like UWER exclusions and buffering issues. It was identified that the noise stemmed from a lack of namespace filtering. Applying an article-only filter significantly smoothed the data however, reverts are inherently unpredictable. Spikes are often caused by bulk actions (like reverting sock puppets) or non-article reverts. It was also noted that smaller wikis naturally exhibit more noise due to lower data volume.
    • The SQL query was shared for further investigation but recommended dedicating data analyst resources for case-by-case spike analysis. @Ahoelzl subsequently tagged @APizzata-WMF to help identify the final root cause of the dashboard implementation issue.

Status Update:

Definition needs: None

• Incremental MW was merged, definition approval confirmed

Implementation needs: Approval

• Test Kitchen is not needed

Active Data Issue:

• The main cause of more noise is probably that this backfill goes back further than previous ones (a year instead of a few months). They aren't planning to adjust it. They've tried a few things to reduce the noise (like changing the time windows) but they were ineffective. The team will see whether any signal from interventions will cut through the noise - perhaps it will, and the noise is acceptable.

Update to definition of Where bad-faith activity is defined based on who reverts the edit - tracking therefore edits reverted by UWERs: added temporary-account-viewer, which is already implemented in git.