Now that we have developed a hierarchical method to analyze experiments across wikis, we want to use it to analyze the help panel experiments, using data from Czech, Korean, and Vietnamese Wikipedias. The main metrics we want to look at are:
A nice-to-have metric would be "non-reverted edits", or some other version of measuring productive edits.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | MMiller_WMF | T206711 [EPIC] Growth: help panel | |||
| Resolved | nettrom_WMF | T206720 Help panel: experiments | |||
| Resolved | nettrom_WMF | T227239 Help panel: hierarchical analysis |
I've completed gathering data for this analysis using the 2019-06 MediaWiki history snapshot. Moving it to "In Progress" as I'll start doing the actual analysis.
@nettrom_WMF -- per our conversation, I've added "edit count/volume" as one of the KPIs we want to look at, in addition to activation and retention. I also added "non-reverted edits" as a something that would be interesting, but not a high priority.
We have a set of slides for sharing this with stakeholders, reassigning to @MMiller_WMF for review.
We have completed this analysis, and I posted some of the results in brief in this update (and will post some more): https://www.mediawiki.org/wiki/Growth/Growth_team_updates#Update_2019-09-27:_help_panel_experiment_results
Now reassigning this back to @nettrom_WMF and putting it into Ready for Development because the final step is to write a fuller report on wiki. We need to decide when and how to do this.
We've completed the analysis but not gotten around to publishing it on-wiki. There were not any substantial findings, instead the team learned a lot about the challenge of running experiments with features that are used by a relatively small percentage of users. Since then, we've repurposed the Help Panel to support Newcomer Tasks, where we've seen a larger impact. It doesn't seem like a good use of time and resources to work on publishing these non-results at this point, so I'm closing this task as resolved.