ReviewStream: improving edit-review tools through a better data feed
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• jmatazzoni
	Oct 24 2016, 11:22 PM

Description

Type of activity

unconference session

Main topic

none

Assigned to

Roan Kattouw, Joe Matazzoni

Description

=== The problem===
The [[ https://www.mediawiki.org/wiki/Edit_Review_Improvements | Edit Review Improvements ]] project seeks to improve the edit-review process generally and, in particular, to provide a more supportive process for good-faith newcomers, a group that research shows have special needs. One focus of the project is [[ https://www.mediawiki.org/wiki/Edit_Review_Improvements/ReviewStream | ReviewStream ]], a machine-readable feed currently in development. To the basic data about recent changes found in RCStream, ReviewStream adds data valuable to reviewers of all types, from welcomers to vandalism fighters. Notably, edit-scoring data from [[ https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service | ORES ]] will help reviewers identify edits by good-faith new users and generally target their work more effectively.

By directly incorporating data that currently has to be looked up in separate processes, ReviewStream is designed to make life easier for creators of downstream edit-review tools and to make their tools faster while encouraging inclusion of features that will help new users.The feed will also benefit from extensive design and user-testing research into the best ways to make ORES' AI predictions clear and helpful to users.

===Expected outcome===
Vandalism fighters and developers – especially those who design or program edit-review tools – will learn about the new feed. They will provide input that will improve the planned features and approach. Some will go on to create new edit-review tools or adapt existing ones to incorporate new feed capabilities.

===Current status of the discussion===
After wide-ranging discussion, we've written a basic ReviewStream spec (see below), and work has begun on shaping the feed's basic architecture. Meanwhile, much design, product research and user testing have gone into finding the right way to talk about ORES predictions and to present ORES scores in ways that conform with users' mental models and goals.

===Links ===
- [[ https://www.mediawiki.org/wiki/Edit_Review_Improvements/ReviewStream | ReviewStream spec ]]
- [[ https://www.mediawiki.org/wiki/Edit_Review_Improvements | Edit Review Improvements (ERI) project page ]]
- [[ https://www.mediawiki.org/wiki/Edit_Review_Improvements/New_Features_for_Special:Recent_Changes | Planned New ERI Features for Special:Recent Changes ]]
- [[ https://phabricator.wikimedia.org/tag/edit-review-improvements-reviewstream/ | Phabricator board for ReviewStream ]]
- Session Notes
-- [[ https://docs.google.com/document/d/1Jv5yBgvFCpvKCa_OwYvMf3vgWNlnq7aqLm3donyGMJA/edit#heading=h.pu94u3k2ybmx | Google Docs ]]
-- [[ https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/ReviewStream | Wiki]]
== Preferred group size ==
12-24

== Any supplies that you would need to run the session ==
- Post-its and markers
- Projector with hookup to a Mac

== Preferred group size ==
15-20

== Interested attendees (sign up below) ==
# your name here

Event Timeline

• jmatazzoni created this task.Oct 24 2016, 11:22 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 24 2016, 11:22 PM

• jmatazzoni renamed this task from ReviewStream: improving edit-reviewing tools through a better data feed to ReviewStream: improving edit-review tools through a better data feed.Oct 24 2016, 11:22 PM

Trizek-WMF added a project: Edit-Review-Improvements-ReviewStream.Oct 26 2016, 1:30 PM

Restricted Application added a project: Collaboration-Team-Triage. · View Herald TranscriptOct 26 2016, 1:30 PM

In the ArchCom meeting just now, I suggested having an IRC meeting about ReviewStream architecture on November 9 in the usual RFC meeting time slot. Do you think that would be useful?

Thanks for the suggestion @tstarling, I'm not familiar with that meeting. I know @Catrope, @SBisson and @Ottomata are trying to put together a non-IRC meeting about this, so will leave the planning to them.

Qgil moved this task from Backlog to Missing proven interest on the Wikimedia-Developer-Summit (2017) board.Oct 27 2016, 1:47 AM

In T149060#2746347, @jmatazzoni wrote:

Thanks for the suggestion @tstarling, I'm not familiar with that meeting. I know @Catrope, @SBisson and @Ottomata are trying to put together a non-IRC meeting about this, so will leave the planning to them.

Let's do that first (to narrow down what approach we want to take), and then have an IRC meeting later?

• jmatazzoni moved this task from Untriaged to External on the Collaboration-Team-Triage board.Nov 3 2016, 1:19 AM

• Capt_Swing added a subscriber: • Capt_Swing.Nov 10 2016, 6:04 PM

Who would be the person facilitating this session? Please assign this task to that person if you are aiming to have this session pre-scheduled. Thank you!

• jmatazzoni updated the task description. (Show Details)Nov 14 2016, 8:46 PM

Qgil assigned this task to • jmatazzoni.Nov 15 2016, 12:29 PM

• Capt_Swing awarded a token.Nov 15 2016, 11:21 PM

• Tbayer awarded a token.Nov 26 2016, 12:21 AM

Any cool outcomes of the Nov. 9th meeting? I'm still not sure where this session belongs, but it seems that the A unified vision for editorial collaboration topic might be just as relevant as AI. It seems that there's a lot of overlap.

"A unified vision for editorial collaboration" was cancelled as main topic. The ArchCom meeting on Nov 9 didn't ave any discussion about this topic?

Re-reading the description of this proposal, there is not really a problem statement, and neither there is a discussion so far. All in all this looks like a presentation of a new project, not a discussion? It might find a good place in the context of the Unconference, I think.

@jmatazzoni Hey! As developer summit is less than four weeks from now, we are working on a plan to incorporate the ‘unconference sessions’ that have been proposed so far and would be generated on the spot. Thus, could you confirm if you plan to facilitate this session at the summit? Also, if your answer is 'YES,' I would like to encourage you to update/ arrange the task description fields to appear in the following format:

Session title
Main topic
Type of activity
Description Move ‘The Problem,' ‘Expected Outcome,' ‘Current status of the discussion’ and ‘Links’ to this section
Proposed by Your name linked to your MediaWiki URL, or profile elsewhere on the internet
Preferred group size
Any supplies that you would need to run the session e.g. post-its
Interested attendees (sign up below)

Add your name here

We will be reaching out to the summit participants next week asking them to express their interest in unconference sessions by signing up.

To maintain the consistency, please consider referring to the template of the following task description: https://phabricator.wikimedia.org/T149564.

In T149060#2879808, @srishakatux wrote:

To maintain the consistency, please consider referring to the template of the following task description: https://phabricator.wikimedia.org/T149564.

done

• jmatazzoni updated the task description. (Show Details)Jan 3 2017, 10:42 PM

• jmatazzoni updated the task description. (Show Details)

• jmatazzoni updated the task description. (Show Details)Jan 3 2017, 10:45 PM

srishakatux moved this task from Unconference candidates to Proposed Unconference Sessions on the Wikimedia-Developer-Summit (2017) board.Jan 4 2017, 1:19 AM

To the facilitator of this session: We have updated the unconference page with more instructions and faqs. Please review it in detail before the summit: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Unconference. If there are any questions or confusions, please ask! If your session gets a spot on the schedule, we would like you to read the session guidelines in detail: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Session_Guidelines. We would also then expect you to recruit Note-taker(s) 2(min) and 3 (max), Remote Moderator, and Advocate (optional) on the spot before the beginning of your session. Instructions about each role player's task are outlined in the guidelines. The physical version of the role cards will be available in all the session rooms! See you at the summit! :)

srishakatux moved this task from Proposed Unconference Sessions to Approved Unconference Sessions on the Wikimedia-Developer-Summit (2017) board.Jan 9 2017, 7:29 PM