Maniphest T328057

Leveling Up: experiment analysis
Closed, ResolvedPublic
Actions

Description

User story:

As the Growth team, I want to understand how Leveling up (part of the Positive Reinforcement project) has changed newcomer behavior, because I want to see if this work was effective and I want to know if any improvements should be made.
As a Wikimedian, I want to make sure new Growth features are working well for newcomers, because then I'll be more accepting of the changes when they are rolled out on my wiki.

Leveling up Phab workboard:
https://phabricator.wikimedia.org/tag/growthexperiments-levelingup/

Positive Reinforcement experiment plan:
https://www.mediawiki.org/wiki/Growth/Positive_reinforcement#Experiment_plan

Acceptance Criteria:

Using the plan outlined in our Positive Reinforcement Experiment Plan and further detailed in our Measurement Specification, provide answers to our Leveling up hypotheses.
Discuss results with Growth team (in synchronous meeting or async)

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		KStoller-WMF	T248330 [EPIC] Growth: Positive reinforcement
		Resolved		KStoller-WMF	T317813 [EPIC] Positive Reinforcement: Leveling Up
		Resolved		nettrom_WMF	T328057 Leveling Up: experiment analysis
		Resolved		nettrom_WMF	T343290 Leveling Up: post-experiment questions & analysis

Event Timeline

KStoller-WMF created this task.Jan 26 2023, 5:56 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 26 2023, 5:56 PM

KStoller-WMF triaged this task as Medium priority.Jan 26 2023, 5:57 PM

KStoller-WMF added a parent task: T317813: [EPIC] Positive Reinforcement: Leveling Up .

KStoller-WMF moved this task from Inbox to Triaged on the Growth-Team board.

KStoller-WMF moved this task from Backlog to Follow-up "Leveling up" tasks (not needed for MVP) on the GrowthExperiments-LevelingUp board.

KStoller-WMF mentioned this in T328060: Leveling up: Instrumentation.Jan 26 2023, 6:15 PM

KStoller-WMF mentioned this in T328386: Leveling Up: community configuration.Jan 30 2023, 10:52 PM

nettrom_WMF added a project: Product-Analytics.Jan 31 2023, 5:54 PM

mpopov moved this task from Triage to Upcoming Quarter on the Product-Analytics board.Jan 31 2023, 6:16 PM

KStoller-WMF mentioned this in T330543: Leveling up: Second design iteration of "Try a new task" dialog.Feb 24 2023, 11:08 PM

KStoller-WMF mentioned this in T248332: Reinforcement: analysis of thanks for newcomer.Mar 26 2023, 10:02 PM

nettrom_WMF moved this task from Upcoming Quarter to Current Quarter on the Product-Analytics board.Apr 11 2023, 5:14 PM

nettrom_WMF changed the task status from Open to In Progress.Jun 15 2023, 9:14 PM

nettrom_WMF edited projects, added Product-Analytics (Kanban), Growth-Team (Sprint 0 (Growth Team)); removed Product-Analytics, Growth-Team.

nettrom_WMF moved this task from Incoming to In Progress on the Growth-Team (Sprint 0 (Growth Team)) board.

nettrom_WMF moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.

Iflorez added a project: User-Iflorez.Jul 18 2023, 8:13 PM

Iflorez moved this task from Backlog to Radar on the User-Iflorez board.

KStoller-WMF mentioned this in T342819: Leveling Up: Improvements to "Get Started" notification .Jul 26 2023, 10:22 PM

KStoller-WMF mentioned this in T343288: Leveling Up: Community configuration for Leveling up notifications.Aug 1 2023, 10:36 PM

Marking off completed work, and I'll add a follow-up task to document any further analysis we want to complete around Leveling up.

Document learnings on wiki: https://www.mediawiki.org/wiki/Growth/Positive_reinforcement#Measurement_and_results

@nettrom_WMF Do you want to complete this, or should I?

KStoller-WMF changed the status of subtask T343290: Leveling Up: post-experiment questions & analysis from Open to In Progress.Sep 6 2023, 10:49 PM

KStoller-WMF mentioned this in T348896: Leveling up experiment analysis: Document significant findings on Wiki.Oct 13 2023, 9:04 PM

In order to break up work into manageable sizes for sprints, I've moved the documentation part of this task into: T348896: Leveling up experiment analysis: Document significant findings on Wiki
And we can consider this task complete.