Page MenuHomePhabricator

Enable parts of ORES extension by default and manage impacts on the RC Page and the RC page Preferences tab
Closed, ResolvedPublic

Description

Per T158225: Enable the ORES good faith and damaging UI by default, on wikis that have these ORES models available (instead of behind a Beta Feature), the ORES extensions should be able to be configured such that it does not add a beta feature-- ORES is on by default. Here are descriptions of the impacts this change will have on the RC Page itself and on the RC Page Preferences tab.

How to handle ORES features on RC Page


- **"r" flags for "needs review"** These are the little R that ORES shows for edits with damaging scores above a certain threshold.
- Because ORES is on by default, the "r" will now display for everyone on RC page.
- HOWEVER, the ORES Sensitivity controls that determine what edits get the "r" will be relocated to Watchlist, as defined in T160475
- AND the ORES Sensitivity thresholds will be conformed to the ERI ORES levels (T160575)

- **Automatic shading of rows based on their damaging score--aka "Classic ORES"**
- This function is not available to RC Page users. Period. This is true whether or not users have opted in to the beta.
- (On Watchlist, shading is now controlled by a new preference (see T160475).

==Changes relating to the RC Preferences tab

  • Hide probably good edits from recent changes This Recent Changes option currently appears only after users have selected the ORES beta, and is unselected by default.
    • The feature will now have two modes:
      1. Users who have opted in to the New Filters beta will not get this function. No edits will be hidden.
        • The option itself will disappear from the RC Preferences tab for these users.
        • However, their setting will be stored, and in the event that the user unsubscribes to the New Filters beta, both the option and the user's setting will be restored.
      2. Users who do not have the New Filters beta will get this function more or less how it works now.
        • HOWEVER, note that the function thresholds will be conformed to the ERI ORES levels, as described in T160475
        • AND, the Sensitivity control that determines what counts as "probably good" will be relocated to the Watchlist Preferences tab, as described in T160575
  • Ores Sensitivity Remove this control from the RC Page preferences. (It moves to Watchlist; see T160475 and T160575.)

Wikis this applies to

The changes described above don't take effect on a given wiki until the beta features are released on that wiki.

  • Portuguese Wikipedia
  • Polish Wikipedia
  • Persian Wikipedia
  • Dutch Wikipedia
  • Polish Wikipedia
  • Russian Wikipedia
  • Turkish Wikipedia
  • Wikidata
  • Czech Wikipedia
  • English Wikipedia

Related Objects

Event Timeline

Change 342643 had a related patch set uploaded (by Sbisson):
[mediawiki/extensions/ORES] Allow the ORES extension features to be 'on' by default

https://gerrit.wikimedia.org/r/342643

@Catrope and @Etonkovidova, as discussed, I've updated this ticket with the Preferences functionality that was described on T158225 (and I resolved that ticket).

Unlike what we discussed, the task above states that "Classic ORES" functionality will not be available on RC page. Here's why I think that makes sense:

  1. Classic ORES used to be turned on via the ORES beta option, which we have eliminated entirely.
  2. In the new scheme, classic ORES functions are available on Watchlist and Contributions by means of the new "Highlight probably damaging edits on Watchlist and Contributions (using ORES)" preference, which is on the Watchlist prefs page.
  3. So, if we want to offer classic ORES on RC page, we'd have to create a new preference on the RC page, And that new preference would be in conflict with the RC Page beta, on the beta page—since you can't have both.

Since our plan is not to support classic ORES in the long run, does it really make sense to create a new preference to maintain it on RC page? Especially when we're migrating all the classic ORES users to RC Filters.

I suppose there is an argument to be made that until RC page is out of beta, classic ORES should be offered as a stop-gap? But classic ORES, too, was a beta. RC Filters is in effect its next incarnation of that beta interface. Are we really worried that old ORES users will want to deselect RC Filters and go back to classic? We clearly informed them on their Talk page that " ...all who have the ORES beta feature enabled will suddenly get a new filtering interface once [we] are ready to deploy it."

And how would we resolve the conflict? I suppose RC Filters could simply trump classic ORES among users who've selected both. But that seems very inelegant. If we move forward with the new interface as sole option on RC Page, is it hard to offer the old option if anyone complains?

(I added some of our colleagues to this ticket: @Trizek-WMF @Pginer-WMF? What do you think?)

  1. So, if we want to offer classic ORES on RC page, we'd have to create a new preference on the RC page, And that new preference would be in conflict with the RC Page beta, on the beta page—since you can't have both.

If you're saying that we don't want to offer the highlighting part of classic ORES on the RC page, that seems fine to me, as it's redundant with RCFilters. However, I think we should still keep the newly-default parts of the classic ORES interface on the RC page. That also wouldn't require anything weird preference-wise, because that stuff isn't triggered by preferences.

I fixed up the description above to a) explain how the former "hide probably good edits" option will work for people who don't choose RC Filters, and 2) to move all the stuff about Watchlist preferences to a new subtask, T160475

If you're saying that we don't want to offer the highlighting part of classic ORES on the RC page, that seems fine to me, as it's redundant with RCFilters. However, I think we should still keep the newly-default parts of the classic ORES interface on the RC page. That also wouldn't require anything weird preference-wise, because that stuff isn't triggered by preferences.

The 'r' is also redundant with the "classic" highlight feature and equally problematic. It is based on the "sensitivity" preference we're about to kill, which has its own set of thresholds that are different from ours. If you highlight all the damaging filters with different colors, you'll see that the 'r' doesn't exactly line-up with one color.

We can keep the "show/hide probably good edits" filter and align it with our thresholds with the understanding that it is less flexible without the sensitivity preference. Where would that be useful? Do we plan to NOT deploy the RC beta to wikis with ORES beta?

@SBisson, good info. Thanks. Two questions;

  • Where does the "r" threshold fall now?
  • How hard would it be to change that threshold so it aligns with one of our categories?

@SBisson, good info. Thanks. Two questions;

  • Where does the "r" threshold fall now?

It is based on the sensitivity preference, and the thresholds are configurable per-wiki. The default values are:

softest: 0.90,
soft: 0.70,
hard: 0.50

If you select "soft", it adds the "r" next to all edits whose score is greater[1] than 0.70. The words and numbers seem backward to me. I thought "hard" would be very strict and only flag edits with damaging score between 0.9 and 1. Maybe I'm missing something.

  • How hard would it be to change that threshold so it aligns with one of our categories?

Not very hard. But it I think would need to be done globally: removing the hardcoded thresholds and use the dynamic stats everywhere.

  1. Unrelated note: RC and Watchlist use ">=" but Contributions uses ">"

@SBisson, good info. Thanks. Two questions;

  • Where does the "r" threshold fall now?

That depends on your sensitivity setting.

  • How hard would it be to change that threshold so it aligns with one of our categories?

We could easily align the three sensitivity settings with the three damaging thresholds, I suppose.

Just to add to the discussion about 'r' - it could be viewed as a first safeguard (especially if the classic ORES highlighting will be removed). With default filters, users see some records on RC page marked with 'r' indicating that there are problems - then they migh go more granular with new RC filters to figure out the extend of damage done by those 'r' marked edits.

Otherwise, looking at the uniform, unmarked set of results in RC page, users have to perform some extra steps to see edits that are damaging.

If you highlight all the damaging filters with different colors, you'll see that the 'r' doesn't exactly line-up with one color.

Although I do not see it as big usability problem, it would be nicer if , @SBisson said to "align it with our thresholds".

Change 342643 merged by jenkins-bot:
[mediawiki/extensions/ORES] Allow the ORES extension features to be 'on' by default

https://gerrit.wikimedia.org/r/342643

Change 342947 had a related patch set uploaded (by Catrope):
[operations/mediawiki-config] Set $wgOresExtension for I63b11eff3a4

https://gerrit.wikimedia.org/r/342947

Change 342947 merged by jenkins-bot:
[operations/mediawiki-config] Set $wgOresExtension for I63b11eff3a4

https://gerrit.wikimedia.org/r/342947

Mentioned in SAL (#wikimedia-operations) [2017-03-15T23:17:55Z] <thcipriani@tin> Synchronized wmf-config: SWAT: [[gerrit:342947|Set $wgOresExtension for I63b11eff3a4]] T159763 (duration: 00m 44s)

@SBisson, the spec for this page has changed considerably. I'm moving this back to In Development so you can please review the spec carefully. (The good news is that it sounds like many of the things I think of as "changes" are just descriptions of the way Roan already arranged things.)

Change 342964 had a related patch set uploaded (by Catrope):
[mediawiki/extensions/ORES] Follow-ups to 980fb74d7: move classic highlights to watchlist

https://gerrit.wikimedia.org/r/342964

Change 342964 merged by jenkins-bot:
[mediawiki/extensions/ORES] Follow-ups to 980fb74d7: move classic highlights to watchlist in 'on' mode

https://gerrit.wikimedia.org/r/342964

Hide probably good edits from recent changes This Recent Changes option currently appears only after users have selected the ORES beta, and is unselected by default.
The control for this option disappears entirely from the RC page. Users who still get this function on RC Page (see below) will get it through a control on Watchlist Preferences. (T160474)

  • Users who have opted in to the New Filters beta will not get this function. No edits will be hidden.
  • Users who do not have the New Filters beta will get this function more or less how it works now.

There is currently 2 distinct preferences controlling this. One for RC and a different one for Watchlist. We are proposing to have one preference control both, which reduces flexibility. I don't mind but I want to make sure it is understood.

I think this is the only thing that is not currently covered by the 2 patches that were merged yesterday.

In T159763#3106700, @SBisson wrote:

There is currently 2 distinct preferences controlling this. One for RC and a different one for Watchlist. We are proposing to have one preference control both, which reduces flexibility.

I do not see the Sensitivity control on Watchlist preferences now (i.e., I don't think there is a loss of flexibility). Are you sure it's there?

In T159763#3106700, @SBisson wrote:

There is currently 2 distinct preferences controlling this. One for RC and a different one for Watchlist. We are proposing to have one preference control both, which reduces flexibility.

I do not see the Sensitivity control on Watchlist preferences now (i.e., I don't think there is a loss of flexibility). Are you sure it's there?

The preferences I'm talking about are: "Hide probably good edits from RC" and "Hide probably good edits from Watchlist". They control the default value of the "show/hide good edits" filter on RC and watchlist respectively.

They are independent. One may choose to show good edits on watchlist and hide them on RC by default. When we merge them, what is selected will affect what you see by default on both RC and Watchlist. That's what I mean by a loss of flexibility or functionality.

This comment was removed by Catrope.
jmatazzoni renamed this task from Add feature flag to enable parts of ORES extension by default to Enable parts of ORES extension by default and manage impacts on the RC Page and the Recent Changes Preferences tab.Mar 17 2017, 9:45 PM
jmatazzoni renamed this task from Enable parts of ORES extension by default and manage impacts on the RC Page and the Recent Changes Preferences tab to Enable parts of ORES extension by default and manage impacts on the RC Page and the RC page Preferences tab.
jmatazzoni updated the task description. (Show Details)

In T159763#3107259, @SBisson wrote:

The preferences I'm talking about are: "Hide probably good edits from RC" and "Hide probably good edits from Watchlist". They control the default value of the "show/hide good edits" filter on RC and watchlist respectively.

They are independent. One may choose to show good edits on watchlist and hide them on RC by default. When we merge them, what is selected will affect what you see by default on both RC and Watchlist. That's what I mean by a loss of flexibility or functionality.

Good point. Thanks @SBisson. After discussion with colleagues, I have rewritten the Description above so that the Hide Probably Good Edits function remains an independent RC Page preference. BUT—this is a new feature—this preference will disappear if the user selects the New Filters beta. (I think Roan has already implemented this.)

We will still move the Sensitivity control to Watchlist. And that Watchlist control will still determine the definition of "Probably Good", as in the description.

@Etonkovidova, I see this is in QA. Please note that I just changed the spec, as described in my previous post.

"r" flags for "needs review"

Two relatively minor issues were filed regarding display of 'r':
T161363: RC filters - 'Category changes' with 'Very likely have problems' are not marked with 'r'
T161060: RC ORES filters do not force a changed state to 'r': it displayed black until page reloaded

the ORES Sensitivity controls that determine what edits get the "r" will be relocated to Watchlist, as defined in T160475

Not done yet - checked in betalabs.

Automatic shading of rows based on their damaging score [...] This function is not available to RC Page users.

It seems to be DONE.

QA recommendation: Resolve

Checked Changes relating to the RC Preferences tab specs on ptwiki - the specs are in place: "Hide probably good edits from recent changes " and "ORES Sensitivity", and ORES highlight option settings have been moved to Watchlist preferences.

Below are the screenshots form ptwiki Watchlist preferences with the settings and ptwiki Recent changes preferences.
Watchlist preferences:

Screen Shot 2017-03-28 at 9.23.20 AM.png (678×1 px, 134 KB)

Recenchanges preferences:
Screen Shot 2017-03-28 at 9.25.09 AM.png (459×1 px, 86 KB)