Page MenuHomePhabricator

Ask for consensus to enable and deploy ORES extension to production
Closed, ResolvedPublic

Description

This task is complete when the ORES extension is enabled and available on a Wikimedia project.

Primary users: patroller, watchlist user

Needs:

  • Quickly review Special:RecentChanges for damage
  • Quickly review Special:Watchlist activity for damage (T122535)
  • Quickly review Special:Contributions for damage (T122537)

Features:

  1. Special:RecentChanges
    1. Edits in RecentChanges that are likely to be damaging are flagged for review,
    2. Edits in RecentChanges that are not likely to be damaging can be filtered out of the list
  2. Special:Contributions
    1. Edits in Contributions that are likely to be damaging are flagged for review
    2. Edits in Contributions that are not likely to be damaging can be filtered out of the list
  3. Special:Watchlist
    1. Edits in Watchlist that are likely to be damaging are flagged for review
    2. Edits in Watchlist that are not likely to be damaging can be filtered out of the list.

Proposed implementation:

  • Roughly replicate the functionality of gadget-ScoredRevisions
  • Cache scores so we can efficiently join when filtering feeds.

Related Objects

Event Timeline

awight created this task.Dec 9 2015, 10:25 AM
awight updated the task description. (Show Details)
awight raised the priority of this task from to Needs Triage.
awight added a subscriber: awight.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptDec 9 2015, 10:25 AM
He7d3r added a subscriber: He7d3r.Dec 16 2015, 4:20 PM
Halfak updated the task description. (Show Details)Dec 17 2015, 3:39 PM
Halfak set Security to None.
awight added a subscriber: Halfak.Dec 28 2015, 11:27 PM

@Halfak
Thanks for writing this design spec!

awight updated the task description. (Show Details)Dec 28 2015, 11:30 PM
Ladsgroup updated the task description. (Show Details)Dec 30 2015, 11:00 PM
He7d3r renamed this task from Deploy ORES extension to production and ask for consensus to enable to Ask for consensus to enable and deploy ORES extension to production.Jan 19 2016, 12:35 PM

I can handle this for you for Wikidata when it's ready.

Excellent. That help will be very much appreciated. :)

We have consensuses ready for deploying it in fa.wp. Here. We are almost good to go :)

Change 269476 had a related patch set uploaded (by Reedy):
Branch ORES for WMF deployments

https://gerrit.wikimedia.org/r/269476

Change 269478 had a related patch set uploaded (by Reedy):
[WIP] Enable ORES on fawiki

https://gerrit.wikimedia.org/r/269478

Change 269476 merged by jenkins-bot:
Branch ORES for WMF deployments

https://gerrit.wikimedia.org/r/269476

greg added a subscriber: greg.Feb 9 2016, 8:23 PM

Just checking here:
The deploy of Extension:ORES will depend on a WMF Labs service? Has this been reviewed by anyone in Ops?

greg added a comment.Feb 9 2016, 8:27 PM

I see Brad already asked this. I would greatly greatly greatly greatly prefer to have ORES in production and not depend on WMF Labs. We've seen two types of fall out from dependencies on Labs.

  1. The usage of the new feature is so much that even the number of requests going Labs brings it down (we had a bit of js loaded on pageviews that did this)
  2. WMF Labs has a hiccup causing the production consumer to fail in bad ways.

How is this being planned for and mitigated in this setup? Also, for the record, we pretty clearly block such cross prod/labs dependencies on the surface, so very good reasons/mitigations are needed.

@greg fair points. I'll follow advisement from ops here. @akosiaris is working on our production systems now. See T106867

@yuvipanda can help with discussions about what is at risk for Labs. FWIW, we're operating at a relatively high capacity at the moment and the system is robust and scale-able. fawiki is a good test case for us since there are few active editors and many of them are already using a tool that queries ORES (in a less efficient way than the extension would).

@akosiaris & @yuvipanda, what do you advise? Should we continue deploying the extension as a beta feature to fawiki while the ORES service is in labs or should we wait until the ORES service can be brought online in "prod"?

As a developer of the extension I need to add that this extension depends on Labs services but when the service is down, no harm would happen to the wiki. Only edits made at the time span wouldn't have their scores stored in the ores_classification table, which means we need to run a maintenance script once a while (the patch of the maintenance script is about to be merged, I'm waiting for some inputs of sysadmins. See T123795)

I just excluded bot edits which reduce the number of requests by 55% for fa.wp. Based on the RC table, this extension would call 1.7 times per minute (for fa.wp) once enabled. I think ores in wmflabs can handle that ;)
I must say: I just giving numbers, I wait for Ops to elaborate more.

It is not technically possible to deploy this extension before ORES is in production, as production cannot talk to *.wmflabs.org domains.

@Ladsgroup, it looks like this task might be done since we have consensus for how to do it -- now we're *doing* it.

Change 294027 had a related patch set uploaded (by Ladsgroup):
Add ORES to extension-list

https://gerrit.wikimedia.org/r/294027

Change 294027 merged by jenkins-bot:
Add ORES to extension-list

https://gerrit.wikimedia.org/r/294027

Change 269478 merged by jenkins-bot:
Enable ORES on fawiki

https://gerrit.wikimedia.org/r/269478

Change 294103 had a related patch set uploaded (by Ladsgroup):
Add ORES to whitelisted beta features.

https://gerrit.wikimedia.org/r/294103

Change 294103 merged by jenkins-bot:
Add ORES to whitelisted beta features.

https://gerrit.wikimedia.org/r/294103

Ladsgroup closed this task as Resolved.Jul 4 2016, 2:03 AM