Page MenuHomePhabricator

Visualize the relationship between the probability of reversion and ores scores
Closed, ResolvedPublic

Description

This is a relatively simple test that RcFilters are having an effect on reversion behavior.

  1. Create samples of namespace 0 edits by week before and after ORES is deployed for each Wiki that adopted ores
  2. Scrape the threshholds that are set for each Wiki for each week from Special:ORESModels
  3. For each wiki plot the probability of reversion as a function of ores scores. Look for jumps at the threshholds when we look after RcFilters changes are released, but no jumps before RcFilters were released.
  4. Try to measure the jumps by modeling the relationship between reversion and ores score and the threshholds.

Event Timeline

Moving this to review because I have been reviewing @Groceryheist's work. I think we should consider this done when his PDF is upload and/or the plots are posted to the relevant meta page.

Finally moving this to done. I have seen the visualizations. Maybe @Groceryheist can post some examples here if he gets the time and explain the results. Ultimately, the paper he is working on will be the conclusion of this line of work.