Page MenuHomePhabricator

Pull and normalize data from enwiki SPI
Closed, ResolvedPublic

Description

Per T415103: WE 4.8.3 Patrolling metrics from enwiki SPI/AIV:

From initial feedback from English Wikipedia patrollers, it seems that Administrator Intervention against Vandalism (AIV) and Sockpuppet Investigations (SPI) are promising starting points, due to their semi-structured nature and their nature as reporting venues for rule-breaking.

This ticket covers the technical work needed to pull and structure this data. It'll probably:

  • Scrape and store all revisions from SPI, reducing the number of needed API calls and allowing for offline processing
  • Apply a heuristic to classify all revisions. This should be transparent and also be saved if possible as all classifications within a definition should never change
  • Manage the ability to also classify all exceptions to the heuristic and save these decisions
  • Perform some kind of data visualization/aggregation and/or export some kind of csv based on classifications (tbd)

Subject to change as the technical problems are worked through.

Acceptance Criteria:

  • This should be re-runnable, as if the data is valuable we would want to re-run for post-intervention metrics
  • As these pages have no major privileged access requirements, this tool/visualizer should be runnable without additional privileges and ideally not require complex engineering support

Event Timeline

See https://gitlab.wikimedia.org/stran/statstbd for all generated data and graphs.

Probably the most interesting graph:

image.png (1,950×1,156 px, 99 KB)

2024 numbers align with the 2024 push to work down the SPI backlog:

image.png (1,958×1,238 px, 392 KB)

T415103: WE 4.8.3 Patrolling metrics from enwiki SPI/AIV Will probably be able to provide more context around this data.

@STran Thanks for the analysis and for the repo: https://gitlab.wikimedia.org/stran/statstbd

We talked about a couple of things:

  • Duplicate investigations (more than one involving the same user) are filtered out
  • We can only capture the time taken to resolve a case after it has been resolved, so this doesn't capture still-open cases

I'd be happy to turn our attention to the AIV analysis now, since we haven't found anything obviously detrimental from the SPI analysis, and we might expect temporary accounts to have more of an impact on AIV

Some notes in case we come back to this:

  • One chart remains WIP, the Open/Closed case counts over time. It shows more closed cases than open cases which seems odd but at this point, ROI on AIAV seems higher.
  • The chart doesn't take into account that scraping starts with a table already in-progress, as before this bot there was another bot managing tracking the work. This is an inconsistency but not enough to explain the ~2k difference in 01/2026 between opened/closed cases, assuming methodology is otherwise correct.
  • The duplicate cases ignored may be reducible. Per Claudia, the bot makes a row for every state the case is in, hence the potential for duplicated rows. It may be possible to identify these duplicate by the existence of an ongoing closed status despite only having one recorded opened case at the moment. This also seems like low yield additional analysis, hence why it wasn't done.
STran renamed this task from Pull and normalize data from enwiki SPI/AIV to Pull and normalize data from enwiki SPI.Feb 4 2026, 1:00 PM
STran updated the task description. (Show Details)