Weekly Updates

• We have been working on prioritizing tasks to improve these models.
  • For the RRLA we have prioritized the following tasks:
    • Test this model on different project family, e.g. wikibook, commons, etc.
    • Research if possible to automatically generate a list of keywords per language, that are related to reverts.
  • For the RRML we have are going to:
    • Investigate if possible to use a simpler mwedittypes functions, to improve the time efficiency on the model without significantly impact the accuracy.
• We have gave a presentation about these models on the Research Showcase (Video)

Weekly Updates

• The Multilingual Model (RRML) would be updated to use a simpler version of mwedittypes to avoid errors.
• Considering the results reported on T340811#9042799 our current recommendation is to use Language Agnostic (RRLA) in all cases (att: @elukey). We are actively working on improve it's performance on IP Edits, we plan to deploy a new version (addressing the anonymous edits issue) within Q1.
• We will continue experimenting with Multilingual approaches (based on LLMs) to build a new generation of Revert Risk algorithms (att: @Trokhymovych)

• Weekly Updates**

• In coordination with @elukey we are updating the RRLA model documentation to facilitate migration from ORES damaging model to RRLA.
• @MunizaA has created an example code to be used on extracting data for (re)training all Knowledge Integrity Supported models (RRLA, RRML and RR Wikidata). We are using that code, to extract data for testing and improving the RRLA.
• We have keep recommending developers and product teams to use the RRLA as default model.
• Also, @Trokhymovych has created a new version for the RRML, that would solves many of the performance issues from the previous models.

Weekly Updates

• We have presented the RRML at KDD'23.
• Some interesting feedback & comments includes:
  • Control for power users on the training and testing dataset.
  • Test with other LLMs
  • In general our model has received very positive comments and we have created awareness about Knowledge Integrity on Wikipedia and specially on it's importance for ML/AI practitioners and researchers when they train and use LLMs

Weekly Updates

• I had a meeting with @JTannerWMF and Dmitry discussing how to integrate the RRLA on the T322083. They are going to contact the ML team to learn more about RRLA integration on the recent changes feed (ORES Extension) cc: @elukey

Weekly Updates

• We are testing different strategies to improve the precision of RRLA on IP Edits.
• We had a meeting with the Moderation Tools team about the Automoderator Project. They are planning to start actively working on this in the following months. We have a conversation with the engineers and designer to explain some technical details about the model.
• I have also been in conversations with Chris Albon, to study the feasibility of using RRLA for "fast vandalism detection" T344537. As you can see in the related task, the model performance fulfill the requirements for that task.

Weeky Updates

• Keep working on testing different setups for RRLA.
• Also we are working on standardizing the process of data collection for training, and considering creating periodical dumps for this data.

Weekly Updates

No major updates this week.

Weekly Updates

• We have updated the Revert Risk Models' documentation on Meta:
  • Adding the most recent results and comparison with benchmarks
  • Including clear recommendation on which models to use for different use cases and the reasons behind it.
  • Sharing the code to reproduce the evaluation experiments.
• I've been tested different training datasets for RRLA (modifying the class distribution on the training data) showing minimal improvements for Anonymous edits.

Weekly Updates

• WME had detected issues with the language coverage of RRLA.
• In collaboration with the ML-team we have fixed the issue, and no new errors has been reported.
• We have defined a methodology for dealing with corner cases (eg. new Wikipedia language editions), establishing default values for new (or unseen) wikis.
• With @fkaelin we are planning to unified some data collection process that is used by the Article Quality model and the RRLA.

Thank you for the work in Q1. This work as planned will continue into Q2 and I'll move it to the relevant column now. Good luck!

Updates

• The Moderation Tools team is running tests and community discussions to implement the Automoderator project, we are in coordination with them to learn about potential areas of improvement for RR.
• There has been some community initiatives to evaluate the quality of the RR models. In T336934 a group of rowiki editors had manually labeled a set of risky revisions. We have analyzed these results, showing reasonable good performance.
• The ML-team is working on integrating RRLA to recent changes feed T348298. We are working on defining the best thresholds for this integration T351897 .
• We have been working with Wikimedia Enterprise to clarify some doubts about the RRLA model T346095

Updates

• The Privacy Engineering team has reviewed the model finding no privacy-related concerns with the model.
• The patch for adding revert risk on the recent changes' feed has been merged. This enables the option of integrate Revert Risk on MediaWiki. Now, I'm working in finding the adequate thresholds for RR scores (T351897) to add the corresponding mediawiki tags.

We are using this tasks as umbrella for reporting improvements and our coordination with products teams regarding the Revert Risk models.
Given that the model showed to be good enough for the Automoderator project, and also would be integrated on the MediaWiki Recent Changes feed (T352217), I think we can resolve this task and report future updates related to revert risk to the EPIC task; T314384