Page MenuHomePhabricator

Educate users about the highlight option and encourage them to try it
Closed, ResolvedPublic

Description

Usability testers found highlighting to be a very helpful feature. In addition to making filter results more meaningful, highlighting also provided a powerful diagnostic that enabled users to answer many of the questions they had about how the filters work, thus helping them master the system more quickly and fully. However, because the Highlight Results button scrolls out of view as users work with the dropdown filter menu, it was also clear that users would benefit from some help in discovering this feature sooner.

solution

A blue dot with a pulsing halo will be shown hovering over the "Highlight results" button inviting users to click. This follows the same pattern as educational popups on Visual Editor (more details in T101481#1504143).

When the user clicks on the button or dot on the "Highlight results" button, a popup panel is shown introducing the feature:

The above should only happen when these conditions are met:

  • When a user who has not clicked the Highlight Results button before opens the Dropdown filter menu for the fifth time, the blue dot, etc. displays.
    • If that user fails to click the Highlight Results button, the blue dot will display again every third time the user opens the Dropdown menu.
  • This applies only to users who have not clicked on the Highlight Results button before. That is, users who already discovered highlight on their own don't need to be introduced to it, and neither do users who have already seen the panel one time.
    • (Users who have been shown the blue dot but not activated the popup panel will continue to see the blue dot when the proper conditions are met.)
  • The highlight feature becomes enabled at the same time as the popup appear.
  • Once the user closes the popup with the "Ok, got it" button, the user will never see the panel again.

Graphical assets

An image of a marker highlighting a row is used in the dialog. The SVG versions for both writing directions are available below:


Details

Related Gerrit Patches:
mediawiki/extensions/WikimediaMessages : masterRCFilters Guided tour: Append the dot to the popup
mediawiki/extensions/WikimediaMessages : masterUse pixel values instead of ems for stillDot size
mediawiki/core : masterRC Filters: Hooks for highlight guided tour
mediawiki/extensions/WikimediaMessages : masterRC filters: highlight guided tour

Event Timeline

@Pginer-WMF, in writing this up for developers I had some questions. Please see below.

When the user hovers over or clicks on the button or dot on the "Highlight results" button, a popup panel is shown introducing the feature:

Is "hovers over or clicks" right here? You originally said "clicks," but I noticed the example you showed me worked on hover. Whatever you think.

The user has not applied highlighting before. That is, users that already used highlight already learnt about the feature by themselves and they don't need to be introduced to it.

Do we want to say "applied highlighting" here, or use the lesser bar of "clicked the Highlight Results button"? I think the latter might be sufficient (and probably simpler to monitor).

,,,has not encountered and dismissed the popup before

"And dismissed" is right here, don't you think? Or is it enough that someone has been shown the popup? The problem with the latter is that the user might simply close the dropdown menu or leave the page without ever having seen the popup.

the popup panel displays -- without first showing the blue dot.

We go straight to the popup, right? No blue dot this time? The point, I think, is to make sure the user sees this at least once.

@Pginer-WMF, in writing this up for developers I had some questions. Please see below.

When the user hovers over or clicks on the button or dot on the "Highlight results" button, a popup panel is shown introducing the feature:

Is "hovers over or clicks" right here? You originally said "clicks," but I noticed the example you showed me worked on hover. Whatever you think.

I'd show this on click. This is also how it actually works for the case of VE (although the initial prototype explored the option of showing it on hover). Showing it on click aligns with the behaviour they'll do later, seems discoverable enough and less likely to accidentally showing/discarding/ignoring the popup.

The user has not applied highlighting before. That is, users that already used highlight already learnt about the feature by themselves and they don't need to be introduced to it.

Do we want to say "applied highlighting" here, or use the lesser bar of "clicked the Highlight Results button"? I think the latter might be sufficient (and probably simpler to monitor).

"Clicked the highlight results button" seems a good-enough approximation. That seems enough.

,,,has not encountered and dismissed the popup before

"And dismissed" is right here, don't you think? Or is it enough that someone has been shown the popup? The problem with the latter is that the user might simply close the dropdown menu or leave the page without ever having seen the popup.

Makes sense. The man idea was to avoid showing it more than once for those completing the process. Considering the process complete when clicking the "ok, got it" button makes sense.

the popup panel displays -- without first showing the blue dot.

We go straight to the popup, right? No blue dot this time? The point, I think, is to make sure the user sees this at least once.

I'd show the blue dot in order to avoid getting in the way if they are doing something else, and to train muscle memory:

  • After using the filters for some time, we make the highlight button to say "I have something to tell you" with the blue dot, but we want to give room for the user to be able to say, "ok, let me finish this important filtering and check that blue dot later". Getting in the way and deliver the message directly instead, can be counter productive (i.e., the user just to scroll in order to add the filter she is looking for).
  • Showing the message when the user actually clicks the button associates the behaviour to the action, training the muscle memory and making it easier to get into the user habits.

@Pginer-WMF writes:

After using the filters for some time, we make the highlight button to say "I have something to tell you" with the blue dot, but we want to give room for the user to be able to say, "ok, let me finish this important filtering and check that blue dot later".

Sounds good. But if the user ignores the dot and doesn't open the popup, should we show the blue dot again? Like, what, every third time she opens the popup? What have you done in the past?

jmatazzoni updated the task description. (Show Details)

I updated the spec -- see above.

@Pginer-WMF writes:

After using the filters for some time, we make the highlight button to say "I have something to tell you" with the blue dot, but we want to give room for the user to be able to say, "ok, let me finish this important filtering and check that blue dot later".

Once we show the dot, I'd propose to leave the dot visible until the user clicks on the button, since it provides a clear way to get rid of it.
We can consider a long period of exposition to mean the user actively ignores the feature (e.g., the user may already know from other wikis and may not be interested in it at all), and remove the dot after that period (e.g., remove the dot after opening the panel the 20th time after the dot was initially shown).

I'd not recommend showing it intermittently (i.e., making it disappear and reappear again) since that will generate more noise and less predictability (i.e., having the dot available when you want to check it).

@Pginer-WMF, I've got a couple of suggested changes to this ticket. Let me know what you think:

  • I'd like to change the text on the popup panel to either 1) simplify by dropping the second paragraph entirely, or 2) make it say "You can combine highlights with filters or use them independently." or 3) include a "Learn more" link to the Highlighting Help page. (1 and 3, I think?)
  • I removed the functionality below from the description. The stuff that goes on in the Conflict and No-effect states is complicated enough, both for us and for the user. Let's get this invitation working for its main purpose and then see if we want to add more use cases.
  • The user reaches a situation where highlighting can be helpful. These situations, which are defined fully under "No-Effect Display States" in T149391, include:
    • Selecting a set of filters where some cancel each other (either because they overlap or they cover the whole spectrum of possibilities).
    • Selecting a set of filters so restrictive that no results are shown..

@Pginer-WMF, I've got a couple of suggested changes to this ticket. Let me know what you think:

  • I'd like to change the text on the popup panel to either 1) simplify by dropping the second paragraph entirely, or 2) make it say "You can combine highlights with filters or use them independently." or 3) include a "Learn more" link to the Highlighting Help page. (1 and 3, I think?)

The options sound good to me, but I'm more in favour of 2. I think the idea that highlight can be used independently of filters is important enough to be mentioned since it enables many possibilities for the user. I'm ok in simplifying the language, an your proposal looks good. A learn more linking to broader documentation (where the user was about to filter the list of recent changes for some review activity) may not get much interest and even for those getting there the specific piece of information still needs to be found.

  • I removed the functionality below from the description. The stuff that goes on in the Conflict and No-effect states is complicated enough, both for us and for the user. Let's get this invitation working for its main purpose and then see if we want to add more use cases.
  • The user reaches a situation where highlighting can be helpful. These situations, which are defined fully under "No-Effect Display States" in T149391, include:
    • Selecting a set of filters where some cancel each other (either because they overlap or they cover the whole spectrum of possibilities).
    • Selecting a set of filters so restrictive that no results are shown..

Sounds good to me.

Thanks @Pginer-WMF. Let's go with this text:

Color Code Your Results
Use highlighting to visually distinguish selected properties.

You can combine highlights with filters or use highlighting independently.

Does that mean you need to create a graphic with this text in it, or is the text message inserted via code?

Quiddity removed a subscriber: Quiddity.Mar 9 2017, 2:16 AM
Pginer-WMF updated the task description. (Show Details)Mar 9 2017, 10:27 AM

Thanks @Pginer-WMF. Let's go with this text:
Does that mean you need to create a graphic with this text in it, or is the text message inserted via code?

No. Text should be real text, not part of the image: renders better, allows to be translated more easily and is usable for screen readers.

What's needed is the image above the text, which was missing in the ticket. I just added to the description, and also here:


Shouldn't we have a "learn more"-like link somethere?

SBisson added a subscriber: SBisson.EditedMar 20 2017, 4:57 PM

I've started looking at this and I'm not sure how to count the "appearances" of the highlight button. Since clicking a filter in the "Active filters area" scrolls the popup to the filter, and it remembers its position when re-opening, it is possible to use the popup for quite some time without ever seeing the highlight button.

Would scrolling the popup up to the button count as an "appearance"? Is there a minimum of the button area that needs to be visible or does it have to be visible for some amount of time for this to count? Would scrolling up and down rapidly count as several appearances or would the popup need to be closed and reopened between each? It gets complicated pretty fast and it may highlight (no pun intended) a real discoverability issue of the button.

Maybe pinning the title bar ( Filters - Highlight button ) to the top of the popup would greatly help discoverability of the feature already. It would also make counting appearances much simpler if a guided tour is still needed.

@SBisson, we said in today't meeting that:

  • we would make the panel "reset" when it's closed, so that on the next open it always starts at the top.
  • Count five of those top openings towards the totals.

Does that give you what you need to finish this?

Change 344163 had a related patch set uploaded (by Sbisson):
[mediawiki/core] [wip] RC Filters: Hooks for highlight guided tour

https://gerrit.wikimedia.org/r/344163

Change 344164 had a related patch set uploaded (by Sbisson):
[mediawiki/extensions/WikimediaMessages] [WIP] RC filters: highlight guided tour

https://gerrit.wikimedia.org/r/344164

SBisson updated the task description. (Show Details)Mar 22 2017, 5:27 PM
Krinkle removed a subscriber: Krinkle.Mar 24 2017, 8:57 PM

Change 344163 merged by jenkins-bot:
[mediawiki/core@master] RC Filters: Hooks for highlight guided tour

https://gerrit.wikimedia.org/r/344163

Change 344164 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMessages@master] RC filters: highlight guided tour

https://gerrit.wikimedia.org/r/344164

Change 346439 had a related patch set uploaded (by Mooeypoo):
[mediawiki/extensions/WikimediaMessages@master] Use pixel values instead of ems for stillDot size

https://gerrit.wikimedia.org/r/346439

Checked in cawiki in beatalabs - all specs seem to be in place:

  • the blue dot appears after the fifth time the drop down panel is open: no interaction is necessary to make the act of opening count. If the panel opens as a result of clicking on the filter capsule - it still is counted too. If some searches are done, so the panel reloads, but it still is open, the count is not increased. Basically, the panel needs to be open and closed to increase the count.
  • Clicking on 'Highlight results' without making any selection is counted as highlight action and the blue dot won't appear again.

Some screenshots:

Waiting on the patch https://gerrit.wikimedia.org/r/346439 to be merged to re-check the blue-dot display implementation.

Change 346594 had a related patch set uploaded (by Mooeypoo):
[mediawiki/extensions/WikimediaMessages@master] RCFilters Guided tour: Append the dot to the popup

https://gerrit.wikimedia.org/r/346594

jmatazzoni closed this task as Resolved.Apr 5 2017, 10:46 PM
jmatazzoni added a subscriber: Mooeypoo.

Looks good. @Pginer-WMF, I still think it might make sense to make the filter panel header sticky, since the blue dot is pretty subtle. @Mooeypoo has essentially written the code to do this. Let us know if you think we should go ahead.

jmatazzoni reopened this task as Open.Apr 5 2017, 10:47 PM

Change 346439 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMessages@master] Use pixel values instead of ems for stillDot size

https://gerrit.wikimedia.org/r/346439

Change 346594 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMessages@master] RCFilters Guided tour: Append the dot to the popup

https://gerrit.wikimedia.org/r/346594

Etonkovidova added a comment.EditedApr 6 2017, 6:16 PM

The fix for the blue-dot is in place - additionally to the usual testing it was checked in IE11 and RTL wiki.

QA Recommendation: Resolve

Looks good. @Pginer-WMF, I still think it might make sense to make the filter panel header sticky, since the blue dot is pretty subtle. @Mooeypoo has essentially written the code to do this. Let us know if you think we should go ahead.

Making the panel header sticky is a good solution to make the highlight functionality more prominent. However, highlighting is an advanced feature and during the initial user research we didn't found issues with the discovery (even without the education feature this ticket describes).
So I'm more inclined to observe how users make use of the tool (paying attention to the highlight discoverability among other aspects), before deciding if we need the feature to be more prominent (and increase the complexity of the panel structure).

Having said that, I'm definitely for a specific ticket to be created to capture the idea, and give it a try in upcoming prototypes to check how that would work.

@Pginer-WMF wrote:

during the initial user research we didn't found issues with the discovery

We have a very different memory on that. My memory is that people did have quite a bit of trouble discovering the Highlight button--precisely because it so quickly scrolls off the screen. This is why we went to all the trouble of inventing this education feature. @dchen may want to chime in with her impressions.

@Pginer-WMF wrote:

during the initial user research we didn't found issues with the discovery

We have a very different memory on that. My memory is that people did have quite a bit of trouble discovering the Highlight button--precisely because it so quickly scrolls off the screen. This is why we went to all the trouble of inventing this education feature. @dchen may want to chime in with her impressions.

Daisy can provide more detail. From the page 27 of the summary report it is indicated that "all participants noticed the highlight feature". It is possible that there were some initial friction in the discovery, but we need to take into account that a sticky toolbar does not make a difference until the user scrolls (not on the exposure to the panel just opened). The behaviour that is more relevant here in my opinion is whether users decide to switch the highlighting mode frequently as they scan the list of filters breaking their flow frequently or not.

jmatazzoni closed this task as Resolved.Apr 18 2017, 12:49 AM