Page MenuHomePhabricator

Improve organization and control for Flow notifications (tracking + ideas)
Open, NormalPublic

Description

While the current notification system fits the needs of casual users (catch up with the activity in the few places they are involved), as users get involved in a higher volume of activities the current system becomes less useful. In particular, this ticket will focus on Flow notifications since it is a representative case, but the solutions proposed can be applied to other kinds of notifications.

This is a general card to describe the problem, explore solutions, and discuss the work we need to solve it. More specific tasks will be created as blockers to this one.

Goal

Allow users to better organise and control notifications:

  • Avoid missed notifications by providing a balance between the overview and the details. That is, communicate the general amount of pending work without overwhelming the user, but give enough information to decide which pieces of work to take.
  • Make notifications more meaningful by aligning the volume of notifications received with the user interest on a given subject. That is, the user will receive more (and more detailed) notifications on the topics that is interested the most.

Scenarios and user needs

To illustrate different needs, let's imagine the following notifications:

  1. Keep me up to date with topics I'm interested in. When the user is interested on a page (and watches it) or participates in a discussion (gets automatically subscribed to a topic), the user will get notifications on new topics (Notification A) and new activity in those the user participated (created or replied to). This usecase seems properly supported with the current solution and we should make sure new solutions are not breaking it.
    1. The only missing use-case is T121138: Enable a way for us to choose whether to autowatchlist each new flow topic
  2. Act on individual items. Notification B represents 30 new topics. Presenting those groupde together helps to avoid the panel to get too crowded with different topics mixed. However, the single notification item does not make it easy to act on each of those 30 new topics: The user cannot check what these are about to evaluate which are the important ones to reply. Once the user access the notification, the user will reach the flow board with the relevant topics highlighted (only for the current session) which forces to reply to all of them in the current session or lose track of the pending ones.
  3. Interest on a topic evolving in time. Users that watch changes on a page may not be interested on new topics, or with any activity at all. Users noticing a new topic may want to immediately subscribe to it. Users overwhelmed by a big group of notifications due to a temporary controversy may want to skip notifications on that topic for a while. Users very interested on a specific board may want to check every single notification. All that may happen for different topics or the same topics at different points in time.
  4. Check what is new about a subject area. Even with bundled notifications, notifications from the same board can be all around the place (e.g., A and E). Users that are interested on checking what i new on a given board may not find it easy to get that overview with other notifications getting in the way. That can result in a loss of efficiency as te user has to move back and forth.

Research and instrumentation

The above issues were identified ad summarised based on input from different people in the team based on reports from the community but a deeper understanding from quantitative and qualitative perspectives will be helpful:

  • User research needs captured at T100650.
  • Instrumentation needs:
    • The value of notifications. Understanding how often is the notification badge ignored (e.g., number of times a user opens the panel when there are pending notifications) we can set a baseline of how much the kind of information notifications provide is useful for users or it just becomes too crowded to even pay attention to it. The global number of ignored notifications per user (or users with more than X ignored notifications) can be also a good metric to focus on reducing.
    • Overview vs. detail. Understanding how much are individual vs. bundle notifications accessed, we can identify when grouping too many notifications together discourages users to go through them.
    • Better conversations. In the long term, people tracking the conversations they care abut should lead to more (and better) participation. Tracking the number of topics with no replies can be interesting to check the overall effect of a better notification system that makes advanced users more productive.

Exploring ideas

We have explored several design directions (you can check the very initial notes if you are interested):

Expand bundled notifications and control the notification volume

  • Prototype. You can expand the first bundle notification and control the volume of the first two, the fifth one, and the whole board from the talk page.

The current notification panel can be be extended to:

  • Allow bundled notifications to be expanded. Bundled notifications can provide an affordance to show the individual notifications they contain. We may want to keep just one notification expanded at a time, visually connect the group and keep the expanded state across different panel openings (since users may want to access topics in sequence).
  • Allow to access options to quickly control topic or board notification patterns. The sound volume metaphor can be used to provide the users control from each individual (topic volume) or bundled notification (board volume). The volume levels can include (as anything, subject to discussion): mute (no notifications), direct mentions (no notifications unless the user is explicitly mentioned), topics the user is involved (to exclude new topics), al activity (notification for topic replies and new topics in the case of boards), and all activity with individual notifications (preventing bundling). In addition, the same control should be provided from the board itself (to be able to revert topics that became muted).


This solution doe not cover the scenario #3, for which some of the solutions below (enhancing the table of contents or providing a specific inbox-like UI can help with)

Control of read/unread status on the board
We can help with he issues from the board itself by:

  • Mark as read only when the user scrolls. In that way even if a user access the board through a bundle of 30 notifications, only a subset of those will be marked as read as the user scrolls. This does not easily support the pattern of making an overview and acting on the most important items leaving the rest for later.
  • Allow to undo the "mark as read". Since marking as read happens as the user scroll, we can provide an affordance to revert that.
  • Make sure the recently notified elements are highlighted in the table of contents. As the table of contents show recent activity (T99785) it can be used to highlight unread topics based on pending notifications.

Never bundle notifications
Bundling is part of the problem, and removing it completely is worth considering. However, it may clutter the notification list by volume and lack of organisation (keeping new topics of the same board all around the place increasing the user back-and-forth navigation).

Provide an inbox-like view for events
Turn the "all notifications" view into a more powerful UI where the user can filter and organise the pieces of pending work. If the user needs to cover are too complex to be served by the notification panel. Instead of stretching the notification panel too much, we can support them with a specific view. This may align well with the idea of Flow feeds where the user can track conversations from different boards.

Crosswatch is a related project which supports displaying and filtering all your notifications, and we can learn about some of their usecases (more details in crosswatch).

Related Objects

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.

Event Timeline

Pginer-WMF updated the task description. (Show Details)
Pginer-WMF raised the priority of this task from to Needs Triage.
Pginer-WMF added a subscriber: Pginer-WMF.
Restricted Application added a project: Collaboration-Team-Triage. · View Herald TranscriptMay 27 2015, 4:31 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Pginer-WMF updated the task description. (Show Details)May 27 2015, 4:33 PM
Pginer-WMF set Security to None.
Pginer-WMF updated the task description. (Show Details)May 27 2015, 5:51 PM
Pginer-WMF updated the task description. (Show Details)May 27 2015, 6:07 PM

Initial feedback: I like the proposed solutions to the problem of notification scale and frequency--bundling and setting "volume" levels. But I'm concerned that adding in all this functionality overloads the notification panel, which is after all supposed to be something you quickly "check", not use for triage. For that reason I like the inbox/watchlist interface proposed at the end.

It would be useful to know what the most common configurations are for users who have customized their settings. Do we have data on this?

But I'm concerned that adding in all this functionality overloads the notification panel, which is after all supposed to be something you quickly "check", not use for triage.

We definitely don't want to support all the ideas. Putting several together may be a convenient way to test them, but we'll be adjusting their prominence based on feedback (e.g., We may find that the volume control works as a metaphor but is not a frequent need and is better not to be at the panel).

That is the reason I suggested to test these ideas with users that just need the most basic needs in addition to the more advanced ones we are targeting the features for. For example, I'd consider the expandable bundle notifications to work if the bundle counter is not adding confusion or distracting the regular user that just wants to view the 3 new topics in the board, while allows the advanced user to quickly view which is the top priority topic from the 50 new ones created on a page he is monitoring.

At the moment all these are experimental ideas to help learning about the problem, several iterations (with the input of our users) are needed before they can be considered solutions.

It would be useful to know what the most common configurations are for users who have customized their settings. Do we have data on this?

I'm not sure which settings are you talking about. Can you elaborate?

Sorry, I was thinking of the settings in the Notifications pref pane (see screenshot).

He7d3r added a subscriber: He7d3r.May 30 2015, 6:05 PM

Thanks for the clarification @Capt_Swing
I'm not aware of any analytics about the current notification settings. It would be interesting to check how many people went through settings to mute a specific kind of notification (or asked for email notification).

DannyH triaged this task as Normal priority.Jun 2 2015, 4:59 PM
DannyH added a subscriber: DannyH.

I like the overall design. I wanted to provide a little feedback of a few problems I've found while testing the prototype, before reading the description of how it's supposed to work.

-I was confused by the "volume" metaphor. I took it literally: the "Notification volume" and loudspeaker metaphor made me think that it was intended to set up the volume of a notification sound that will play when receiving a notification, until I actually read what the options do. I think it would be best to avoid a metaphor that is frequently used in a literal way in similar contexts.

-I freaked out when clicking somewhere on the panel and two horizontal grey bars appeared in the middle of the list. I did not know how I had triggered them, how to dismiss them, nor what it meant.

I didn't realize that the three elements right below the one I clicked where new ones, and are supposed to be part of a bundle; I only noticed that the overal structure of the list had changed, not that new elements had appeared. The elements within bundle should be more visually distinct from their surrounding and with a more obvious style connecting them - and separating them from the elements outside it, maybe with a different background color or a different indentation.

As for Capt_Swing concern that per-topic settings override the Notifications panel, that could be solved by adding a new entry at the "Notification volume", and making this the initial value for new topics:

  • "Use settings from preferences (current value for Flow: Web, Email)"

This way, we power users could still set particular preferences for special topics, but quickly change the global settings for everything else (all the topics that we have not tweaked explicitly).

Johan added a subscriber: Johan.Jun 18 2015, 4:28 PM
Johan removed a subscriber: Johan.
Legoktm moved this task from Backlog to Needs plan on the Notifications board.Jul 6 2015, 7:26 AM
Jay8g added a subscriber: Jay8g.Jul 16 2015, 4:17 AM
Trizek-WMF added a comment.EditedJul 21 2015, 7:44 PM

The deployment of Flow on 3 user talk pages on French Wikipedia during WIkimania raise the noise caused by notifications. Experienced users prefer to be warn of a new conversation by their watchlist instead of receiving a new notification. Filtering notifications would be great to change the idea of using the watchlist (not easy for beginners), or a way to opt-out new subjects notification when posted on talk pages.

A similar system exists at Loomio, for each topic:


Announcement/docs at https://www.loomio.org/d/mpPvDNiI
and a previous version at https://www.loomio.org/d/phOJ4kXn/request-for-feedback-discussion-notification-volume-settings

DannyH renamed this task from Improve organisation and control for (flow) notifications to Improve organization and control for Flow notifications (tracking + ideas).

Re: discussion in a meeting about Phabricator's own "notifications" control options... (email/notify/ignore)
In many respects, goodreads.com is my current ideal model - just look at this delicious malleability! (I know developers are very wary of adding preferences (because extra complexity, and testing, and maintenance) but users tend to love them, especially the broad diversity of powerusers in highly-complex projects, like ours. Especially in linguistically oriented projects, where a controllable-firehose of updates, of many different types (which different editor-archetypes will each find of different value), is the best we can hope for.

ggellerman added a subscriber: ggellerman.

Removing Design Research project. Please feel free to re-add if work on this task resumes.

Pginer-WMF updated the task description. (Show Details)Sep 30 2015, 6:47 PM
AVRS added a subscriber: AVRS.Nov 16 2016, 10:08 PM