The work on New-Editor-Experiences aims to improve the acquisition and retention of new editors on medium-size Wikipedias. Although the specific projects have not been defined yet, we already know their general goals. Being able to measure the impact of these efforts is key.
It is also important to start early defining and capturing these metrics about acquisition and retention for new editors to establish a proper baseline (i.e., considering seasonality over the year, etc.). Especially in the case of retention, where waiting a reasonable time is required to consider an editor as no longer active.
Some of the requirements I'd expect for metrics on acquisition and retention:
- Visualisation of the evolution over time for specific wikis (e.g., this month the retention on Korean Wikipedia has increased 20%).
- Be able to focus on specific user groups that did a given action (e.g., those that clicked on a button we provide with a new tool)
- Be usable for A/B testing (e.b., the retention for the users that participated in the experiment is twice as high as those with the regular experience).
Having these tools ready would put us in a great position to start any kind of project in the New-Editor-Experiences area, being it a big project or a set of small experiments. It is possible that there is some previous work that can be reused/adapted about the definition, capturing and/or representation of these metrics.
Background
Proposed metrics from the annual plan:
Target 1: In emerging communities, the second-month retention rate of new accounts increases
Target 2: In emerging communities, the revert rate for edits by new accounts and IPs decreases
Planning
This Google doc (WMF only at the moment).