While certain instances of vandalism or policy violations can be relatively easily detected using Machine Learning/AI-based tools, most sophisticated patrolling requires human judgment. The goal of this project is to study the feasibility of using AI-based tools to simplify the way that patrollers discover and prioritize content that requires their attention.
In this task, we are going to create and test a recommender system that matches revisions with patrollers. To do this, we are going to rely on existing ML/AI tools, such as the Revert Risk and Peacock (also known as Tone Check) language detection model scores, to identify revisions that require patrolling. Unlike the Automoderator or Tone Check tools that focus on cases with really high scores (e.g., score > 0.95)—i.e., in cases where the vandalism or policy violation is very clear—in this case, we are going to consider milder scores (e.g., 0.5 < score < 0.9), and try to identify a set of experienced users that can review that content.
The input for this RecSys will be: revision_id, score_p, where score_p is the probability returned by some of the aforementioned models. The output of the RecSys will be a set of N users that are likely to review the revision.
For ground truth, we are going to use the revision history and try to predict which user patrolled a given revision.
The goals for Q1 FY25-26 are to:
[] Create the dataset
[] Define metrics
[] Build a RecSys algorithm
[] Run an experiment to understand the complexity of the problem
This work is highly related to proposals by @fkaelin and @pablo. I'm going to coordinate with them to avoid duplicating efforts.