Proposed enhancements to the Notifications Page include search and filtering tools designed to aid users in processing their notifications (see a mockup). It's assumed these tools will be most valuable to users who receive a lot of notifications.
Figures for the aggregate volume of notifications are available. But we don't have good information on the volume of notifications received by individual users. Or, to rephrase that slightly, we don't know what proportion of users receive a significant volume of notifications. This information would be helpful as we attempt to gauge the need for the proposed sorting and searching tools.
In thinking through the goals and methodology for this research, the following was discussed:
- Since editors don't necessarily edit every day, a week seems like a good period to measure. I.e., how many notifications do people get in a week?
- Since Active Users represent less than 0.1% of registered users, it's understood that the vast bulk of users get few notifications. So the analysis here should focus on those users who get the most messages. I.e., a graph showing that 99.9% of users get fewer than 5 messages/week is not the point. More helpful is to know how many users get more than 25 messages/week (the number that the flyout menus can display)? And among that group, what is the distribution--how many get a little more and how many get a lot more?
- Another data point worth having would be to know how many messages go unread. I'm not sure what approach to this would be the most helpful. A few that come to mind include:
- A simple count of the distribution of unread messages, presumably at some point in time during the measured week but without restricting when the messages were generated.
- A count that looks only at the messages generated during the week. So, of messages generated during the week, how many remained unread at the end of the week. We'd still want a distribution – how many users have a few, how many users have a lot?
- Some middle ground that would look measure the number of messages generated during a preceding month.
- Neil says it would be easier to restrict the study to a few wikis. We discussed the following:
- Enwiki and Commons
- Some wiki or wikis that implement Flow most thoroughly. Neil will consult Nick and Benoit on the best choices there.
- First draft of the report is due Feb 12.
- Final draft due Feb. 17.
Find the meeting notes where this project was discussed here: https://etherpad.wikimedia.org/p/notif-metrics