Page MenuHomePhabricator

New Impact Module: experiment analysis
Open, MediumPublic


User story:

  • As the Growth team, I want to understand how the new Impact Module has changed newcomer behavior, because I want to see if this work was effective and I want to know if any improvements should be made.
  • As a Wikimedian, I want to make sure new Growth features are working well for newcomers, because then I'll be more accepting of the change when they are rolled out on my wiki.

Positive Reinforcement Experiment Plan:

Acceptance Criteria:

Event Timeline

KStoller-WMF created this task.

Just noting that this initial experiment analysis is leading us to look into activation and the current Impact Module empty state further: T330614: New Impact module's empty state on mobile: research spike

The team noted a few questions that would help us decide on next steps:

  • Comparing the funnel of users who tap on empty impact module, then press "See suggested edits". (If there is a marked decrease in users who click on "See suggested edits" in the new impact module, that could help explain the lower activation rate.)
  • Can we correlate the activation data with the device information (ex. Mobile screen sizes)?
  • We are using ~6 weeks of data, and there have been several changes during that time. If we compare the first 3 weeks to the second 3 weeks, does the difference in activation rate between the experiment groups look about the same?