I think we should go with a dedicated tab in preferences because number of preferences for ORES will gets increased in near future (for example article quality models support and "meta ores" preferences)

CC: @Catrope @jmatazzoni

OK so if we do a dedicated tab, a critique might be "if we give ORES its own tab, why not give everything its own tab!?"

As @Ladsgroup points out, ORES will have an increasing number of setting *and* each new features that uses ORES' predictions will probably benefit from common setting and a common space for settings. E.g. the ORES settings used by RC Filters could exist on this preferences page. I'm a fan of allowing users to manually tune their own thresholds and this could be the right place to make the controls available to users.

In T167910#3355082, @Halfak wrote:

OK so if we do a dedicated tab, a critique might be "if we give ORES its own tab, why not give everything its own tab!?"

ORES influences so many products and services that it desserves its own tab IMHO.

As @Ladsgroup points out, ORES will have an increasing number of setting *and* each new features that uses ORES' predictions will probably benefit from common setting and a common space for settings. E.g. the ORES settings used by RC Filters could exist on this preferences page. I'm a fan of allowing users to manually tune their own thresholds and this could be the right place to make the controls available to users.

I absolutely agree.

Change 360918 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[mediawiki/extensions/ORES@master] Move preferences hooks to a dedicated file

https://gerrit.wikimedia.org/r/360918

Change 360918 merged by jenkins-bot:
[mediawiki/extensions/ORES@master] Move preferences hooks to a dedicated file

https://gerrit.wikimedia.org/r/360918

Change 361298 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[mediawiki/extensions/ORES@master] Move all of ORES preferences to a dedicated tab

https://gerrit.wikimedia.org/r/361298

Adding user-notice: create a new tab in Preferences, dedicated to ORES, is a big move. People should be aware of it.

When (and where? All wikis?) will this happen? I see it's been marked for deployment this week, but I don't see anything on the wikis where wmf.7 has been deployed.

The tag was for a small back-endish patch. It's not merged yet (and needs @Catrope to sign it off I guess)

@Halfak @Ladsgroup

Should we add designers to this discussion? Thanks!

I want to review this as well. If there's a patch, what does that mean? Can I see this someplace?

@jmatazzoni I couldn't find a public place to put it, I add a screenshot here. If you want to test, it just needs someone cherry pick the change in localhost somewhere, @Catrope might be able to do.

Please take a pause

I said when last we met that I'd write this ticket up. So thanks for the help and your input, but please slow down on this for now. I need to make a complete review of all the preference changes required for Recent Changes and Watchlist before we can move forward.

Why do I need a thorough review? Because Global Collab team is mandated this quarter (in our goals) to 1) take the New Filters for Edit Review out of beta and 2) put the New Filters UI and tools onto the Watchlist (in beta, initially). Meanwhile, we've reworked the Recent Changes page already so that users can save settings and defaults right on the page. So most of the RC preferences on the RC Preference page are unnecessary/redundant. Which is why we're going to be removing them.

Soooo, I need to review the entire Preferences scheme so that it makes sense 1) for users who have the new default (i.e., most people), 2) for New Filters beta users, who will be getting the new UI on Watchlist as well as additional tools on RC page, and 3) for users who've opted out of the new default and are still using the old UI. So this will be a little complicated. Thanks for your patience.

The idea of adding a separate ORES tab

I don't understand this proposal, on a number of levels:

• Right now, the existing "classic" ORES options are not shown to Recent Changes users who have the New Filters beta, because classic ORES is incompatible with the new display in a number of ways. The same will be true for Watchlist users as soon as we roll the beta out there in the coming weeks. So what would be on an ORES tab at present? Would it show up only for users who have opted out of the new UI?
• In my view, ORES per se is not an end-user tool. It's an engine that powers many end-user tools, such as Recent Changes and Watchlist. End users looking to change their settings for those pages will—rationally and because we've trained them to—look on the Preference pages for Watchlist and RC page. (Or better, as discussed above, they won't even have to go to the Preferences pages, because we're making Preferences redundant whenever possible.)
• Amir points out that new ORES products are coming. Good. How will they be implemented for users? Some may at first be beta features in their own right. In which case they'll be listed on the beta tab. Others might be included in the New Filters beta, which is also on the beta tab. Others will be appropriate—either out of the gate or after a beta phase—for end-user tools like Watchlist and RC Page. In which case, they'll be listed on those preference pages, if necessary. What am I missing? If there's a bigger picture I'm not seeing, please help me out.

An aside about ORES product strategy: The Good Faith and Damaging tests have graduated out of beta. ORES is now turned on by default, and these tests will shortly be part of a standard toolset for all users on relevant wikis. This is a good thing. It also means that users of classic ORES will be a dwindling breed. (If they're not–if users opt out of the new UI in large numbers—that will indicate a problem that we'll need to address.)

So as new ORES products come out, if they're appropriate for Watchlist and RC page, it seems to me that the sensible strategy will be for the Rev Scoring and Collab teams to work together to bring these new capabilities out in the new UI. That way, our efforts will be mutually reinforcing. Otherwise, we'll not only be working at cross purposes, but we'll be making users choose between having cool new ORES features and having a powerful UI with lots of very useful new capabilities. Does that make sense? Or is there a reason I'm not seeing why new features would be developed in the old style?

In T167910#3414308, @jmatazzoni wrote:

• Right now, the existing "classic" ORES options are not shown to Recent Changes users who have the New Filters beta, because classic ORES is incompatible with the new display in a number of ways. The same will be true for Watchlist users as soon as we roll the beta out there in the coming weeks. So what would be on an ORES tab at present? Would it show up only for users who have opted out of the new UI?

There is wikis where there is no ORES. So no ORES, no tab. :)

• In my view, ORES per se is not an end-user tool. It's an engine that powers many end-user tools, such as Recent Changes and Watchlist. End users looking to change their settings for those pages will—rationally and because we've trained them to—look on the Preference pages for Watchlist and RC page. (Or better, as discussed above, they won't even have to go to the Preferences pages, because we're making Preferences redundant whenever possible.)

At the moment, it is confusing: if you want to change prediction threshold for Recent Changes, where predictions are most visible, you have to go to the Watchlist tab.
Have a single tab is not redundant with having preferences directly accessible from the different pages where predictions are used (RCs, watchlist...). On the contrary, it would be the central dashboard for global preferences about ORES, a central point to opt-out for users who wish it. A new tab will also welcome new settings for new ORES features, as Amir mentioned. It anso can be the place where settings for specific ORES services would be set. We can create links from Watchlist tab or Recent Changes tab to ORES tab.

• Amir points out that new ORES products are coming. Good. How will they be implemented for users? Some may at first be beta features in their own right. In which case they'll be listed on the beta tab. Others might be included in the New Filters beta, which is also on the beta tab. Others will be appropriate—either out of the gate or after a beta phase—for end-user tools like Watchlist and RC Page. In which case, they'll be listed on those preference pages, if necessary. What am I missing? If there's a bigger picture I'm not seeing, please help me out.

I'm not sure to understand the point concerning Beta features. When Crosswiki Notifications were on Beta, we have added options in the Notifications preferences that were only visible for Beta users. Have Beta settings is possible. Maybe the more global question would be to redefine all the preferences tabs (which are a bit messy)?

An aside about ORES product strategy: The Good Faith and Damaging tests have graduated out of beta. ORES is now turned on by default, and these tests will shortly be part of a standard toolset for all users on relevant wikis. This is a good thing. It also means that users of classic ORES will be a dwindling breed. (If they're not–if users opt out of the new UI in large numbers—that will indicate a problem that we'll need to address.)

So as new ORES products come out, if they're appropriate for Watchlist and RC page, it seems to me that the sensible strategy will be for the Rev Scoring and Collab teams to work together to bring these new capabilities out in the new UI. That way, our efforts will be mutually reinforcing. Otherwise, we'll not only be working at cross purposes, but we'll be making users choose between having cool new ORES features and having a powerful UI with lots of very useful new capabilities. Does that make sense? Or is there a reason I'm not seeing why new features would be developed in the old style?

Like @Halfak says (iirc), "we don't do product". ORES is a tool used by product teams to add a big plus to other tools. Some product teams may have to work on ORES integration to fit to a particular tool, but ORES remains an autonomous tool. Separated settings are then relevant, because as said, ORES is not only a Watchlist/RC page thing.

This new tab is an ORES team project decision to take, because they offer a tool for all other teams, products and features.

Amir points out that new ORES products are coming. Good. How will they be implemented for users? Some may at first be beta features in their own right. In which case they'll be listed on the beta tab. Others might be included in the New Filters beta, which is also on the beta tab. Others will be appropriate—either out of the gate or after a beta phase—for end-user tools like Watchlist and RC Page. In which case, they'll be listed on those preference pages, if necessary. What am I missing? If there's a bigger picture I'm not seeing, please help me out.

When there is a feature, there is also settings related to that. So for example if another team introduces new features related to ORES, which will be beta, the option to enable it will be in the beta tab but the other options (such as thresholds, etc.) need a place and they will sit on the ORES tab.

As an example, we've released a "draft quality" model that could/should be used in Extension:PageCuration. That will likely need very different settings than the edit quality models.

There's been some related discussion in T167911. We all agree that the "new filters" UI is an improvement, and that "saved filter settings" is a sufficient replacement for our ORES preferences pane. In fact, as @jmatazzoni said above, the two approaches actually conflict because the "prediction threshold" will not be respected by RC filters.

We're a bit lost as for what to do about Special:Contributions and no-script browsing, however. As I understand it, RC filters will not be applied to these pages, so we're stuck with the simple filters including ORES checkboxes. It doesn't seem like we can get rid of the preferences, without providing some other way to configure the user's preferred threshold? If we're stuck keeping preferences, can we fix the language so it doesn't seem to conflict with RC filters? @Trizek-WMF @pau your opinions will be of value here!

Collab-Scoring meeting minutes resolving this task are recorded in https://www.mediawiki.org/w/index.php?title=Topic:Tvsudsw9odbah87e&topic_showPostId=tvsudsx3arjn0tsa#flow-post-tvsudsx3arjn0tsa

Change 361298 abandoned by Ladsgroup:
Move all of ORES preferences to a dedicated tab

https://gerrit.wikimedia.org/r/361298