Page MenuHomePhabricator

Use Jade as a repository for existing ORES counterexamples
Open, LowPublic

Description

I hear that some tools like Huggle have internal storage for when users judge that an ORES score is incorrect. This is valuable data for ORES engineering, and we'd love to help get that data exported into Jade.

Event Timeline

awight created this task.Jun 13 2018, 11:06 AM
Restricted Application added subscribers: RichSmith, Aklapper. · View Herald TranscriptJun 13 2018, 11:06 AM
Petrb added a subscriber: Petrb.Jun 14 2018, 9:19 AM

Can you give me some examples how would you interact with this JADE thing? For example, an edit on enwiki is scored by ORES as good quality, but reverted by user in Huggle, meaning it's probably a bad quality edit. What should I do, which API would I call and how the API call format would look?

Petrb triaged this task as Normal priority.Jun 14 2018, 9:19 AM

@Petrb Thanks for the questions! We're still finalizing the schema for judgments, but I'll post the most recent thoughts in T179301: Document JADE judgment structure. It would be great if you wanted to give feedback on the schema before we deploy, of course.

JADE judgments are JSON stored in a new MediaWiki namespace, so it will be edited using the normal content APIs, like action=edit.

Copying from the other task, I've started an explanation here: https://www.mediawiki.org/wiki/JADE/Schema

I found some references to the action=patrol API in Huggle source, but didn't realize until now that it isn't used in practice:

SELECT
  params["action"], count(*)
FROM
  wmf_raw.ApiAction
WHERE year = 2018
  AND month = 6
  AND day = 20
  AND hadError = false
  AND
    userAgent regexp '^Huggle'
GROUP BY
  params["action"]
LIMIT 1000000
;
query   196666
watch   37
delete  2
login   79
block   3
rollback        421
compare 39482
edit    425
thank   5

I guess the current storage for "patrolled" really is internal—would you mind pointing to some of the code that does that?

Petrb added a comment.Jun 25 2018, 9:00 PM

@awight that's because you are probably searching only English wikipedia where patrolling was disabled long time ago, here is very old note from config page:

# patrolling-enabled will make huggle patrol all edits that were marked as good edits
# this feature slows huggle down and creates extra traffic when the feature isn't
# supported by local wiki installation, so make sure to properly tune this variable according
# to wiki config

# English wikipedia has patrolling disabled
# per https://en.wikipedia.org/w/index.php?title=Wikipedia:Village_pump_%28news%29&diff=9146943&oldid=9146404
# so huggle can't use it
Petrb added a comment.Jun 25 2018, 9:01 PM

The change was made on 2005 so I don't know if it's still valid :P

/me crunches on popcorn.

Thanks for the pointer! My query was actually against all wikis, but only for a 24-hour period. It's fine if this integration is disabled everywhere, I'm just curious to understand the current workflows.

If a revision is reviewed in Huggle and is not reverted, I assume you're sharing some type of data with other Huggle users to prevent them from redundant review of this same change?

I assume you're sharing some type of data with other Huggle users to prevent them from redundant review of this same change?

Huggle has a feature named "HAN", that uses IRC to share instant data between patrollers. Source is at huggle/huggle_ui/vandalnw.cpp.

It looks like clients can't communicate any type of "PATROLLED" message, indicating that a change has been reviewed and is good?

Vvjjkkii renamed this task from Use JADE as a repository for ORES counterexamples to d4aaaaaaaa.Jul 1 2018, 1:04 AM
Vvjjkkii raised the priority of this task from Normal to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
RichSmith renamed this task from d4aaaaaaaa to Use JADE as a repository for ORES counterexamples.Jul 1 2018, 6:37 AM
RichSmith lowered the priority of this task from High to Normal.
RichSmith updated the task description. (Show Details)
RichSmith added a subscriber: Aklapper.
Harej moved this task from Inbox to Radar on the Jade board.Jul 6 2018, 3:26 AM
awight renamed this task from Use JADE as a repository for ORES counterexamples to Use JADE as a repository for exisiting ORES counterexamples.Nov 14 2018, 11:08 PM
awight lowered the priority of this task from Normal to Low.

This is sort of on hold, we've decided that we're only interested in Jade-like collaborative judgments, and there's no point in just bulk importing data of potentially lower quality.

awight moved this task from Radar to Product on the Jade board.Nov 14 2018, 11:10 PM
awight removed a subscriber: awight.Mar 21 2019, 4:03 PM
Harej renamed this task from Use JADE as a repository for exisiting ORES counterexamples to Use Jade as a repository for existing ORES counterexamples.Apr 15 2019, 10:26 PM
Harej updated the task description. (Show Details)
He7d3r added a subscriber: He7d3r.Sun, Sep 1, 2:17 PM