Page MenuHomePhabricator

Deploy pilot of Jade to a small set of wikis.
Open, MediumPublic

Description

The Scoring-platform-team is planning to deploy Jade -- a human judgment tracking system useful for keeping tabs on ORES, helping patrollers coordinate with each other, and generating new data for training ORES. We'd like to start by piloting the system in a small set of interested wiki communities.

Good candidate wikis for deployment are communities who are using ORES for patrolling work. Patrollers in these wikis are interested in giving feedback to ORES predictions.

We'll deploy the MVP to a trial set of wikis.

  • Identify collaborators on 2-4 small wikis to help us communicate about the roll-out of Jade
  • Load the extension on the wikis
  • Work with local patrolling tool developers to add integration to Jade
  • Review results and iterate.

The MVP includes:

  1. A full featured API for creating and modifying Jade entities (useful for tool integration)
  2. An entity page UI for creating and modifying Jade entities directly from MW
  3. Integration with Special:RecentChanges and Special:Watchlist
  4. The ability to search Jade entity pages

For more information about Jade, see https://mediawiki.org/wiki/JADE. To explore what a "Jade entity" looks like, see TODO.

Related Objects

StatusSubtypeAssignedTask
OpenNone
OpenNone
OpenNone
InvalidNone
InvalidNone
Resolvedawight
Resolvedawight
Resolvedawight
Resolvedawight
ResolvedHalfak
ResolvedNone
InvalidNone
ResolvedHalfak
Resolvedawight
ResolvedHarej
DuplicateNone
OpenNone
Resolvedawight
Resolvedawight
DeclinedNone
ResolvedReedy
ResolvedLadsgroup
ResolvedLadsgroup
Resolvedmmodell
ResolvedNone
ResolvedNone
Resolvedhashar
ResolvedNone
Resolvedawight
DeclinedNone
DuplicateNone
DeclinedNone
DeclinedNone
DeclinedNone
DuplicateNone
DuplicateNone
DuplicateHarej
InvalidNone
DuplicateHalfak
DuplicateNone
InvalidNone
InvalidNone
OpenNone
OpenACraze
OpenNone
ResolvedACraze
DeclinedNone
DuplicateACraze
ResolvedHalfak
ResolvedHalfak
ResolvedACraze
ResolvedACraze
OpenNone
ResolvedHalfak
OpenNone
OpenACraze
Resolvedkevinbazira
ResolvedACraze
OpenNone
OpenACraze
ResolvedNone
ResolvedHalfak
DuplicateHalfak
Resolvedkevinbazira
ResolvedACraze
ResolvedACraze
ResolvedACraze
OpenNone
OpenNone
OpenACraze
Openkevinbazira
OpenHalfak
ResolvedACraze
ResolvedACraze
OpenNone
Resolvedkevinbazira
ResolvedHalfak
DuplicateHalfak
Resolvedkevinbazira
Resolvedkevinbazira

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Tagging for internal techcom discussion. Is the documentation on the implementation of JADE somewhere?

Note to self:

awight added a comment.Jun 6 2018, 4:02 PM

@daniel Thanks for taking a look. The implementation is quite simple at the moment, it's just a namespace "JADE" with validation to enforce a schema and some business logic. See https://phabricator.wikimedia.org/diffusion/EJAD/

The hope is that curation and patrolling tools like Huggle will be integrated so they create a JADE page where all patrolling decisions made on a wiki entity are collected. As you saw, revisions can be assessed as a "Diff" or as a snapshot of the page, "Revision". There's also a path for judgments about a "Page", and only one will be created, e.g. https://en.wikipedia.beta.wmflabs.org/wiki/JADE:Page/188553

My napkin currently shows that an upper bound success case in which all patrolling goes through JADE will result in c. 2.5M new JADE: pages per year. We don't expect success to come quickly, of course. The current plan is to deploy our first iteration now, and consider how to store data in a more scalable form in the longer term, so that's the question we'd like to receive advice about.

Thanks!

awight added a comment.Jun 6 2018, 4:09 PM

@daniel Oops, I didn't realize which task we were in. Here's the discussion about scalability so far: T196547: [Epic] Extension:JADE scalability concerns

Joe added a subscriber: Joe.

Change 440124 had a related patch set uploaded (by Awight; owner: Awight):
[operations/mediawiki-config@master] [DNM] Enable Extension:JADE in production

https://gerrit.wikimedia.org/r/440124

daniel moved this task from Inbox to Watching on the TechCom board.Jun 13 2018, 8:22 PM

@daniel We're sort of in limbo now, implicitly blocking on potential TechCom discussion. Please let us know if there's a set date to discuss, to help us plan.

@awight In our last session, TechCom decided that we should keep an eye on this, but there is no action required at this point.

JADE si self-contained enough that the only aspect of it that lies within TechCom's responsibility seems to be scalability. If people start to create JADE pages for every new revision (semi-)automatically, we have a serious problem. But at the moment, it seems like supervision by the local communities should be enough to prevent this.


So far the official TechCom position.

Personally, I'd prefer for JADE to get its own separate wiki. Or access to the JADE namespace could at least be severely restricted.
Besides performance, I'm also worried about this being abused to smuggle bad things past edit filters. But then, if nobody sees it anyway...

@daniel Thanks for the helpful notes. We'll probably come back to TechCom for more discussion in a few months, once we see how integrations behave in practice.

Personally, I'd prefer for JADE to get its own separate wiki. Or access to the JADE namespace could at least be severely restricted.

This is an interesting idea, which we've discussed a bit. So far, we think that JADE is something like a structured talk page, and should be governed by the same permissions and communities as create the content on each wiki. For example, an admin on commons who is banned on enwiki should be able to participate in JADE curation on commons, but be excluded from discussion in enwiki JADE. As for restricting permissions, we can always take this step if it proves necessary.

Joe added a subscriber: BBlack.EditedJun 18 2018, 10:43 AM

I would like this to wait for a review by the DBA and Traffic teams.

Specifically: how badly would we be affected by a growth of the page table that a massive adoption of this would cause? Should we make all JADE pages uncacheable so that we don't pollute the cache?

I would at least think we should exclude bots from editing/creating such pages.

@jcrespo @BBlack any thoughts?

(Please note that the SRE summit is underway this week, so you will likely need to wait next week for an answer/advice)

Joe added a subscriber: jcrespo.Jun 18 2018, 10:44 AM
Joe added a comment.Jun 18 2018, 10:46 AM

See also T196547 where the discussion should probably continue

awight claimed this task.Jun 18 2018, 1:58 PM
awight moved this task from Active to Pending deployment on the Scoring-platform-team (Current) board.
awight moved this task from Pending deployment to Review on the Scoring-platform-team (Current) board.
awight updated the task description. (Show Details)Jul 3 2018, 10:16 AM
awight added a subscriber: Harej.
Harej moved this task from Inbox to Blocked on the Jade board.Jul 5 2018, 10:02 PM
Harej changed the task status from Open to Stalled.Jul 23 2018, 5:02 PM
Harej changed the status of subtask T199520: Determine which wikis will get Jade and when from Open to Stalled.
awight renamed this task from Deploy JADE extension to production to [Blocked] Deploy JADE extension to production.Aug 6 2018, 4:48 PM
awight added a project: Epic.
awight renamed this task from [Blocked] Deploy JADE extension to production to Deploy JADE extension to production.Aug 28 2018, 10:47 PM
awight changed the task status from Stalled to Open.
awight removed a project: Patch-For-Review.

Unblocking as I have steps to take, then we'll resubmit for another round of technical and community review.

awight updated the task description. (Show Details)Aug 28 2018, 11:10 PM
awight changed the status of subtask T199520: Determine which wikis will get Jade and when from Stalled to Open.Nov 1 2018, 5:22 PM
awight moved this task from Radar to Inbox on the Jade board.Nov 14 2018, 11:02 PM
awight moved this task from Inbox to Product on the Jade board.
Harej triaged this task as Medium priority.Nov 14 2018, 11:12 PM
SPoore added a subscriber: SPoore.Dec 14 2018, 8:20 PM
Zache added a subscriber: Zache.Jan 15 2019, 1:11 PM
ElanHR added a subscriber: ElanHR.Feb 13 2019, 6:47 PM
Halfak moved this task from Untriaged to Epic on the Scoring-platform-team board.Apr 2 2019, 9:31 PM
Harej renamed this task from Deploy JADE extension to production to Deploy Jade extension to production.Apr 15 2019, 10:20 PM
Harej removed awight as the assignee of this task.
Harej updated the task description. (Show Details)
Harej removed a subscriber: awight.

Change 440124 abandoned by Awight:
[DNM] Enable Extension:JADE in production

https://gerrit.wikimedia.org/r/440124

Harej removed a subscriber: Harej.Jul 4 2019, 9:25 AM
Wargo added a subscriber: Wargo.Jul 8 2019, 7:15 PM
Halfak renamed this task from Deploy Jade extension to production to Deploy Jade extension MVP to production.Aug 6 2019, 7:50 PM
Halfak updated the task description. (Show Details)
Wargo removed a subscriber: Wargo.Jan 6 2020, 6:42 PM
Halfak renamed this task from Deploy Jade extension MVP to production to Deploy pilot of Jade to a small set of wikis..Feb 28 2020, 8:14 PM
Halfak updated the task description. (Show Details)