GrowthExperiments sends notifications to newcomers to reinforce them and improve engagement and re-engagement (as part of the feature Positive_reinforcement#Levelling_up ). In the past the impact of the notifications was analyzed in an experiment form, comparing the activation of users receiving the notification vs users not getting it.
As part of the SDS 2.1.6 hypothesis Growth wants to validate to different use cases of xLab:
- A long-lived clickthrough metric to measure the product health of Growth notifications
- An A/B test experiment to explore different messaging for the notifications
There are three notifications to instrument:
| Notification | 2023 audience | 2023 copy | 2025 audience | 2025 copy (T400118) |
| Get started | 0 suggested edits and less edits than max* | Try a suggested edit and start building your editing skills | 0 edits | Curious about editing Wikipedia? your knowledge joins a community reaching millions. |
| Re-engage | — | — | 0 suggested edits, at least 1 edit and less edits than max* | Thanks for editing. Try suggested edits to develop new skills, and learn about best practices |
| Keep going | 1 suggested edit and less than max* | You’ve made X suggested edits. Keep going to learn more new skills. | 1 suggested edit and less edits than max* | Great progress on your edits! Keep exploring more suggestions to grow your skills. |
*The maximum is defined by GELevelingUpGetStartedMaxTotalEdits for both the 2023 get started notification and the 2025 re-enage and by GELevelingUpKeepGoingNotificationThresholdsMaximum for the keep going one.
Acceptance criteria
- Assess how suitable is the result for product health monitoring
- Assess how suitable is the result for conducting experiments
- The experiment results can be browsed in xLab
Nice to have
- The CTR of each notification is computed using xLab and shown in some conventional chart
Release plan
- Release to some beta wiki and test events are validated and ingested
- Release to testwiki and test e2e the integration with https://mpic.wikimedia.org/
- Release to a group of pilots and test e2e the integration with https://mpic.wikimedia.org/