Status update (November 29th, 2016)
November 29th, 2016

(The post was copied from https://lists.wikimedia.org/pipermail/ai/2016-November/000118.html)


This is the 30th and 31st weekly update from the revision scoring team that
we have sent to this mailing list. We accidentally skipped a week again.

New development:

  • We added a new "lowest" sensitivity level to ORES review tool. This new sensistivity level will only flag edits that ORES is very confident are actually damaging[1].
  • We applied the MediaWiki standard color palette to Wikilabels[2]
  • We generated a manually censored public dataset of spam/vandalism/attack pages[3]. This will help others to develop spam, vandalism and attack page detection models. See the publication of the dataset[4].
  • We've implement color-based confidence reporting for ORES damage detection[5]

Maintenance and robustness:

  • We updated the version of OOjs-UI that gets bundled with Wiki labels[6] and moved the static assets to a new repositiory[7]
  • We fixed an issue in the recscoring library[8] that caused ORES to return invalid JSON and rendered the UI useless[9].


  • We gave a 3 minute presentation on the state of ORES to Victoria Coleman, the WMF's new CTO[10].
  • We performed a basic analysis of Wikipedia article quality trends using the dataset we released a few weeks ago[11]. We'll have a more substantial analysis soon.
  • We made a post on the ORES review tool talk page[12,13] detailing how we plan to incorporate a new filtering strategy into the ORES review tool. Please join the discussion there.
  1. https://phabricator.wikimedia.org/T150224 -- Add "Lowest" ORES sensitivity for fpr=0.1
  2. https://phabricator.wikimedia.org/T151119 -- Apply ui standardization color palette to Wikilabels
  3. https://phabricator.wikimedia.org/T150307 -- Create manually vetted dataset of spam/vandalism/attack pages
  4. https://dx.doi.org/10.6084/m9.figshare.4245035
  5. https://phabricator.wikimedia.org/T144922 -- Visually report damaging confidence
  6. https://phabricator.wikimedia.org/T151222 -- Update bundled OOJS-ui with Wikilabels
  7. https://github.com/wiki-ai/flask-oojsui
  8. https://phabricator.wikimedia.org/T150961 -- ORES ui is broken (text field disabled)
  9. https://github.com/wiki-ai/ores/issues/177
  10. https://phabricator.wikimedia.org/T150544 -- ORES (a 2-3 minute presentation)
  11. https://phabricator.wikimedia.org/T151214 -- Basic analysis of Wikipedia quality using monthly predictions
  12. https://phabricator.wikimedia.org/T150858 -- Post about ORES review tool including ERI filters
  13. https://www.mediawiki.org/wiki/Topic:Tflhjj5x1numzg67

Aaron from the Revision Scoring team

Written by Halfak on Jun 3 2017, 6:16 PM.
Principal Research Scientist

Event Timeline