Page MenuHomePhabricator

Enable product analytics to use revision risk to assess edit quality in feature analyses
Closed, ResolvedPublic

Description

T314384 will introduce new machine learning models to make patrollers aware of suspicious edits so that they can decide whether said edit ought to be reverted.

This task involves the work of reusing the "revision risk" score T314384 will assign to every edit, across all Wikipedias, in a way that enables Product-Analytics to use this "revision risk" data to assess edit quality in the feature analyses they do.

Story

  • As the Editing Team's product manager, I need to be able to assess the quality of edits people are making with the contribution tools/experiences we are responsible far, so that we can answer questions like Should this contribution tool/experience be made more/less widely available?, What – if any – changes might we need to make to this tool/experience in order to improve the quality of changes people use it to publish?, and How does the quality of changes people are using a given tool/experience to publish compare to the quality of changes people are publishing with the previous/legacy tool/experience?.
  • As a Data Scientist...

Requirements

@MNeisler to fill in


@MNeisler raised this idea in the 13 Sep meeting (private doc) with @diego, @nayoub, and @ppelberg.

Event Timeline

The model is already available, check here how to use it: T314385#8496547

@MNeisler Is there a specific ask for the Research team here? (I'm asking b/c I'm reviewing our tasks in our backlog/staged for prioritization purposes.)

@leila Thanks for the ping! No further ask for the Research team here.

Based on discussions with the product analytics team, we can use the revert prediction API model described in https://meta.wikimedia.org/wiki/Research:Develop_a_ML-based_service_to_predict_reverts_on_Wikipedia#APIs in its current format. This is currently being discussed as another method to help assess edit quality in future analyses in addition to other metrics such as edit revert rates.

I'll reach out to @diego if I have any questions regarding use but I think we can go ahead and close this task.

MNeisler claimed this task.