In order to test whether newer users find the Review Changes module compelling, and crucially whether it drives retention, we would like to run an A/B test, where some users see the module and some don't. Because this feature doesn't currently compel users to make edits we can instead track daily participation, to see if it drives users to come back to the site, or their Homepage.
In particular, we would like to test our new user journey idea, which is that users with 20 - 100 mainspace edits are invited to start tracking changes to the pages they have edited. As such, we would show these users a version of Review Changes which has the following sources:
- Pages the user has substantively (non-minor, non-revert) edited
- Pages on the user's watchlist
- Pages related to pages the user is interested in
We may want to analyse what users would see, given this edit count and list of sources: T426612: Analyse activity on pages users have edited when they make their 20th mainspace edit
Because we don't want to confuse new users with a separate, temporary, dashboard - before we've joined the two together - and to help us make a decision about how important it is to combine the two, we could surface Review Changes in the Newcomer Homepage with a simple link to Special:PersonalDashboard, and provide a quick way to get back there:
Mockup
Design
TBD - T431084: Design a Homepage module inviting users to view Review Changes
Invite
We will want to invite users to interact with the Review Changes module in some way. It's not yet clear how we will do that. We could show the user some kind of popup the first time they see the Review Changes module on their Homepage.
PersonalDashboard considerations
Because users will be seeing this module from the Homepage, and because we don't want to distract them with the other Dashboard modules:
- We will want to only display the Review Changes module (and Homepage link), hiding the others. (We can run future A/B tests introducing some of those modules again)
- We don't want to show these users the PersonalDashboard onboarding popup.
- We don't want to provide these users with the personal tools menu link.
- We would need to add a temporary, unlockable, module to the Newcomer Homepage that links out to Special:PersonalDashboard.
Further testing
We chose 20 edits as the unlock milestone because by this point users are likely to have a decent number of edits to review as part of this module. We could consider other unlock milestones. At one extreme we could imagine displaying the module as soon as any edit happens to a page the user contributed to.
Acceptance criteria
- If a user has made 20 - 100 edits, they see a new module on Special:Homepage.
- Module design TBD
- Selecting this module takes users to Special:PersonalDashboard, with only the Homepage and Review Changes modules visible.
- On PersonalDashboard, the Homepage module displays a link back to the Homepage with the title Return to Homepage.
