Page MenuHomePhabricator

PatruBot is too sensitive
Closed, ResolvedPublic

Description

PatruBot is reverting too many good edits. It needs to be more specific (higher precision) and less sensitive (lower recall).

Event Timeline

Generally, I think you want to set very strict criteria on a bot that automatically reverts vandalism.

E.g. with this query, you can see that setting a confidence threshold at 0.96 "probability" of damaging, you can expect to get 90% precision (wrong 1/10th of the time) and 17.7% recall (catches 17.7% of the most damaging edits):

https://ores.wikimedia.org/v3/scores/eswiki?models=damaging&model_info=statistics.thresholds.true.%27maximum%20recall%20@%20precision%20%3E=%200.9%27

Patrubot is no longer performing antivandalism tasks (it's been replaced by SEROBot from @Superzerocool). Perhaps this task is no longer relevant and could be closed.

awight claimed this task.
awight subscribed.

Thanks for the good news!