I added some design proposals for the onboarding steps in the ticket description. Depending on the final name of the feature we'll adjust the wording, but I wanted to focus and get feedback on the proposed process and approach.

In T144457#2798387, @Pginer-WMF wrote:

I added some design proposals for the onboarding steps in the ticket description. Depending on the final name of the feature we'll adjust the wording, but I wanted to focus and get feedback on the proposed process and approach.

Your suggestions make me think about how do you plan to provide access to the documentation? there is some suggestions:

• have access to the documentation from the Beta features pages, like it is at the moment for other feature, and as you suggest for feedback on your mockups
• in T147054: Design: banners for help pages directing users to either the beta help or the old RC Page help

I don't know where we should discuss and decide about that.

In T144457#2802589, @Trizek-WMF wrote:

I don't know where we should discuss and decide about that.

I think both entry points for documentation make sense. One is part of the beta feature information, the other is reachable while using the tool. I think both should point to the same place to avoid fragmentation of feedback.

There is already a suggestion concerning the feedback page: T150959: Integrate a feedback page link in Recent Changes Beta filters

Another aspect on this ticket is the beta feature name and description.
Some of the design goals I considered when looking for a proposal:

• Communicate the purpose and what makes the approach special. Highlighting the end goal of the feature (help you make better reviews) and the means to achieve it (AI predictions, filters).
• Avoid jargon. Mention ORES to help existing users make the connection between the old and new beta feature, but don't make it the main element (since it won't mean much to users not already familiar with it).
• Keep it simple. Use a short name, users can remember and refer to it later (e.g., when providing feedback).
• Help find the feature. the feature will affect different pages, but we may want to emphasise Recent Changes, since better filtering will be provided. Pointing to the Recent changes page could help users find it (during research it was not clear everyone knows how to get there).

Proposal
An example proposal is shown below:

Review predictions
Filters and highlights potentially good and damaging edits. ORES machine learning algorithms help you review new edits at the [Recent Changes page] and more.

@jmatazzoni, any thoughts?

Pau asks:

@jmatazzoni, any thoughts?

Thanks for getting the ball rolling on this. I like your icon, but to my ear the idea of "Review predictions" is still a little too focused on ORES. Although ORES and its predictions is certainly an important aspect of what we're selling, it's now incorporated into a larger package that improves filtering and edit review in a variety of ways. Users won't be getting only ORES, as now, when they opt in; they'll get a whole new filtering interface--at least on RC page and soon to be others. Although Aaron has a quibble with the term, I think we can have it both ways a bit by calling this something along these lines:

• Intelligent filtering
• Intelligent edit review
• Intelligent edit review, with ORES predictions

That lets us explain that the package includes the ORES tests and also an improved interface and highlighting.

The project is called "Edit review improvement", so I think it should be mentioned in the product title. "Intelligent edit review" is an interesting idea.

I'm not that fan of mentioning ORES in the Beta feature title, because people will not understand what it is. I prefer to have ORES mentioned in the Beta feature description, and people will have more information when they click on the "learn more" link.

It makes sense to focus on the filtering part. I'm not convinced about using "intelligent", "smart", etc. since these sound a bit empty (similar to "super", "hyper", "mega"... ) and generic (as in "smartphone", "smart cities", etc.).

Thinking on filters and edit review as the core concepts, would it make sense to just name the feature "Filters for edit review" or "Review filters"?

I created a prototype to illustrate different variants of the beta feature invite, to try different strategies. to use any of the versions open the link and click on the "Recent Changes" link from the Wikipedia sidebar:

• Inline with the existing filters
  
• Bottom panel
  
• Pulsing and expanding dot
  
• Beta features popup
  

Beta features popup have my voice (and Inline with the existing filters as a backup).

From the feedback we got, the popup version seems to work well and it is also aligned with what we did on similar invites (so it has some extra points for consistency). I'll add details to the ticket description to have an updated proposal as a reference for discussion.

I suggested text for the beta description, based on the feature title we approved in the team meeting. See that proposal at T149385

I added some details to the description, including the animated gif to be used in panels showing the filters appearing gradually: F5053416

Regarding some of the aspects we discussed:

Extending the invite to refer to the beta page I explored the possibility of pointing to the beta feature page (see F5053320). I think it makes sense to provide users with a way to provide feedback or go back to the previous version if needed. However, we can only provide this while the feature is still provided as a beta (i.e., it is not the default). Thus, we should describe two cases (while the feature is in beta and when it becomes the default), or making sure we remove such note before making the feature the default. If this sounds right I can update the description.

A more explicit "don't show again" action I was exploring options to provide a more explicit way to discard the popup invite (something along the lines of F5053360). I have two concerns with this route. On the one hand, making the discard option longer and more prominent, we make the main action a bit less prominent. This requires to read more to make a decision. On the other hand, we may be making a promise we cannot hold to: since we may not be able to guarantee not showing it again when users move across wikis.

As an alternative I was thinking to keep the process implicit with some lower intensity reminder. Users that don't want to provide a clear answer, can be provided a less distracting reminder until a more explicit decision is made. This will prevent the popup to appear repeatedly as they try to complete their current task which may involve accessing several times the Recent Changes (e.g., to apply different filters).

Initially	Once it is ignored
Popover shows initially with two options. Any of them would make the popup to go away and not show again (at least for the current wiki)	When the popover is ignored (clicking elsewhere, not in either option) a reminder remains. A blue dot will be shown below the "beta" link. Hovering the beta link will show the popover again for the user to make a final decision. Once the user selects one of the options in the panel, the blue dot won't be shown again.

Any thoughts?

I rewrote this task to spread some of the things that were discussed here to other tasks. Namely:

• Implement the beta preference option for RC Filters project
• Show a message on the RC page introducing the new features

@Pginer-WMF, I put this in the Design column and assigned it to you because there are a couple of small items I've requested. Find them highlighted in pink with your name on them in the Description.

Once you've cleared those tasks, please unassign yourself and put this in RFP. Thanks.

In T144457#3054387, @jmatazzoni wrote:

@Pginer-WMF, I put this in the Design column and assigned it to you because there are a couple of small items I've requested. Find them highlighted in pink with your name on them in the Description.

I'll comment about those below:

• If the user neither implements the feature nor clicks the "no thanks" link (instead, say, navigating away from the page), then the popover is still permanently dismissed. !![@Pginer-WMF -- is that right? Just show one time, even if there's no evidence that the user saw the popover? Should we show one more time?

I think here it is better to err in the side of caution. Considering that the tooltip is prominent enough, different wikis have different settings (users will see the tooltip anyways once per wiki), and that the use of some filters causes a page reload, we should be cautious on showing a non-critical message more than needed.

One option I explored is to use a lower intensity reminder to indicate that there is a pending decision to be made without getting too much in the way. I shared an idea to support this below, but I'm not sure the number of people ignoring the tooltip would be worth the effort:

Initially	Once it is ignored
Popover shows initially with two options. Any of them would make the popup to go away and not show again (at least for the current wiki)	When the popover is ignored (clicking elsewhere, not in either option) a reminder remains. A blue dot will be shown below the "beta" link. Hovering the beta link will show the popover again for the user to make a final decision. Once the user selects one of the options in the panel, the blue dot won't be shown again.

• [@Pginer-WMF --The "confirmation message" doesn't include an actual confirmation. Can you please add something to the design saying something like: "New settings confirmed"?]

Makes sense.

Regarding the language, I prefer to talk about the "new filters" being enabled, rather than the settings being changed. I find the former better aligned with the previous messaging and the user interests (getting new capabilities) while the later seems more of a technicality (how is that change represented in the software):

Sounds good. Un-assigning you to this task and moving it to RFP.

I can work on this but I would like @Pginer-WMF and/or @jmatazzoni to confirm that the description is up to date with the decisions made in the conversation.

In T144457#3201835, @SBisson wrote:

I can work on this but I would like @Pginer-WMF and/or @jmatazzoni to confirm that the description is up to date with the decisions made in the conversation.

I don't think the description is up to date. In T144457#3057454 I made some proposals based on @jmatazzoni remarks in the description. He may want to check my comments and update the description accordingly (or discuss further).

Another detail that is not captured in the description is whether we want to initially target only users with the ORES beta feature initially or not (targeting any user that visits recent Changes instead). This was discussed, but I'm not sure there was a firm decision and whether should be part of this task or a specific one.

Thanks for your follow-up Pau. I updated the description to clarify the two points that were questioned:

1. Regarding whether we will show the user the popup more than the one time specified: not at this time. If we don't see sufficient beta opt in, we'll step things up.
2. Regarding the change in wording, use the wording in the new graphic Pau provided, below (I've also updated the Description with this).

@SBisson, you should have everything you need, I think.

Thanks @Pginer-WMF and @jmatazzoni. Looks good. I should be able to work on it tomorrow.

I use this command in the js console to reset the state during testing. Make sure you are logged in.

(new mw.Api()).saveOptions( { 'rcenhancedfilters-seen-invite': 0, 'rcenhancedfilters': 0, 'rcenhancedfilters-seen-tour': 0, 'rcenhancedfilters-show-invite-confirmation': 0 } ).then( console.log, console.error );

@Pginer-WMF do you have the graphic for the confirmation step?

In T144457#3215413, @SBisson wrote:

@Pginer-WMF do you have the graphic for the confirmation step?

The confirmation step uses the "feedback" icon from the repo in the standard blue (Accent50, or #3366CC). It is the same icon we used in the feedback link at the bottom of the filters panel.

In case that a custom version is needed as an image, this can be used:

Change 350553 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/WikimediaMessages@master] Rc Filters beta feature invitation tour

https://gerrit.wikimedia.org/r/350553

This is what the two steps look like with the patch above. I kept the same basic style for both (margin, text size, text color) and similar to the other tours we already have (welcome and highlight). Let me know how they should be tweaked or if it's important that they look identical to the prototype.

Also, 3 seconds for the confirmation step to disappear feels short to me. It competes for attention with the new filters that have just been loaded. The first time I only noticed it when it went away and I was wondering what it was that I just missed. I don't know if it's a valid concern. More people should try it...

@SBisson, how can we see this to test the timing? Also, on the colorful invitation, please also bold and capitalize the word "New" and put the whole beta name in quotes, like so: "New filters for edit review."

In T144457#3218334, @jmatazzoni wrote:

@SBisson, how can we see this to test the timing? Also, on the colorful invitation, please also bold and capitalize the word "New" and put the whole beta name in quotes, like so: "New filters for edit review."

I've changed the text.

For the timing, someone would have to download the patch and show it to you on their laptop.

In T144457#3217462, @SBisson wrote:

Also, 3 seconds for the confirmation step to disappear feels short to me. It competes for attention with the new filters that have just been loaded. The first time I only noticed it when it went away and I was wondering what it was that I just missed. I don't know if it's a valid concern. More people should try it...

I have not tried this in beta using T144457#3214087 yet (the patchset was not merged yet). If we feel the message gets in when there is a lot of change and there is little time to read it we can do two things: (a) add a short delay (e.g., 0.5 seconds) so that the popup appearing is perceived as a separate change, and (b) increase a bit the time it is visible (e.g., to a total of 4-5s).

Change 350553 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMessages@master] Rc Filters beta feature invitation tour

https://gerrit.wikimedia.org/r/350553

Checked in enwiki betalabs (in monobook skin too) - all specs seem to be in place; as per @SBisson comment - the timeout for the second step confirmation is more than 3 sec.

One thing though: if a user opts in to the beta feature via the inviting popover (clicking on "Enable the new filters"), the user will not get the tour "Introducing: New Filters for Edit Review (beta)..." on the RC page.

The contents of the two tours popovers are different:

@jmatazzoni Do we need to provide "Introducing: New Filters for Edit Review (beta)..." tour for all users who opt for RC filters? Or it's ok do not give it to users who opt in via "New beta feature available for this page" popover?

QA Recommendation: Product should weigh in

In T144457#3226483, @Etonkovidova wrote:

One thing though: if a user opts in to the beta feature via the inviting popover (clicking on "Enable the new filters"), the user will not get the tour "Introducing: New Filters for Edit Review (beta)..." on the RC page.

This is intentional. although the text is not exactly the same, the purpose of both is to introduce the feature. For the invite process, the feature is introduced before they access it. When the feature is enabled explicitly, the feature is introduced once the user gets there. It makes not much sense to introduce the feature twice in my opinion.