Page MenuHomePhabricator

Help panel: hierarchical analysis
Open, MediumPublic

Description

Now that we have developed a hierarchical method to analyze experiments across wikis, we want to use it to analyze the help panel experiments, using data from Czech, Korean, and Vietnamese Wikipedias. The main metrics we want to look at are:

  • Activation
  • Retention
  • Edit count/volume

A nice-to-have metric would be "non-reverted edits", or some other version of measuring productive edits.

Event Timeline

Restricted Application added a subscriber: revi. · View Herald TranscriptJul 3 2019, 11:26 PM
nettrom_WMF triaged this task as Medium priority.Jul 8 2019, 5:03 PM
nettrom_WMF moved this task from Triage to Next Up on the Product-Analytics board.
nettrom_WMF edited projects, added Growth-Team (Current Sprint); removed Growth-Team.

I've completed gathering data for this analysis using the 2019-06 MediaWiki history snapshot. Moving it to "In Progress" as I'll start doing the actual analysis.

nettrom_WMF moved this task from Next Up to Doing on the Product-Analytics board.Jul 10 2019, 5:57 PM
MMiller_WMF updated the task description. (Show Details)Aug 10 2019, 12:46 AM

@nettrom_WMF -- per our conversation, I've added "edit count/volume" as one of the KPIs we want to look at, in addition to activation and retention. I also added "non-reverted edits" as a something that would be interesting, but not a high priority.

MMiller_WMF updated the task description. (Show Details)Aug 10 2019, 12:48 AM
MMiller_WMF set Due Date to Aug 23 2019, 7:00 AM.
Restricted Application changed the subtype of this task from "Task" to "Deadline". · View Herald TranscriptAug 10 2019, 12:48 AM

Analysis piece is done, next up is reporting & sharing with stakeholder

kzimmerman moved this task from Doing to Review on the Product-Analytics board.Sep 18 2019, 9:40 PM

We have a set of slides for sharing this with stakeholders, reassigning to @MMiller_WMF for review.

We have completed this analysis, and I posted some of the results in brief in this update (and will post some more): https://www.mediawiki.org/wiki/Growth/Growth_team_updates#Update_2019-09-27:_help_panel_experiment_results

Now reassigning this back to @nettrom_WMF and putting it into Ready for Development because the final step is to write a fuller report on wiki. We need to decide when and how to do this.

Moving this to current quarter to be done together with other documentation tasks.

nettrom_WMF moved this task from Inbox to FY 2019-20 on the Growth-Team board.Apr 13 2020, 5:51 PM
nettrom_WMF moved this task from Review to Current Quarter on the Product-Analytics board.