Page MenuHomePhabricator

Measure key aspects for New Editor Experiences initiatives
Closed, DeclinedPublic

Description

The work on New-Editor-Experiences aims to improve the acquisition and retention of new editors on medium-size Wikipedias. Although the specific projects have not been defined yet, we already know their general goals. Being able to measure the impact of these efforts is key.

It is also important to start early defining and capturing these metrics about acquisition and retention for new editors to establish a proper baseline (i.e., considering seasonality over the year, etc.). Especially in the case of retention, where waiting a reasonable time is required to consider an editor as no longer active.

Some of the requirements I'd expect for metrics on acquisition and retention:

  • Visualisation of the evolution over time for specific wikis (e.g., this month the retention on Korean Wikipedia has increased 20%).
  • Be able to focus on specific user groups that did a given action (e.g., those that clicked on a button we provide with a new tool)
  • Be usable for A/B testing (e.b., the retention for the users that participated in the experiment is twice as high as those with the regular experience).

Having these tools ready would put us in a great position to start any kind of project in the New-Editor-Experiences area, being it a big project or a set of small experiments. It is possible that there is some previous work that can be reused/adapted about the definition, capturing and/or representation of these metrics.

Background

Proposed metrics from the annual plan:

Target 1: In emerging communities, the second-month retention rate of new accounts increases
Target 2: In emerging communities, the revert rate for edits by new accounts and IPs decreases

Planning

This Google doc (WMF only at the moment).

Event Timeline

Restricted Application added subscribers: revi, Aklapper. · View Herald TranscriptNov 23 2017, 3:41 PM
Ptrpov added subscribers: flimport, QChris, Qgil and 60 others.
Ptrpov added a subscriber: Ptrpov.
cscott added a comment.Dec 1 2017, 8:43 PM

How do we undo the duplicate-merge? Just re-open the closed task?

Qgil removed a subscriber: Qgil.Dec 7 2017, 11:30 AM
MBinder_WMF triaged this task as High priority.May 3 2018, 8:16 PM
MBinder_WMF moved this task from Triage to Next Up on the Product-Analytics board.May 3 2018, 8:17 PM
Neil_P._Quinn_WMF closed this task as Declined.Jun 28 2018, 8:27 PM

This has been superseded by a variety of different metrics efforts around Audiences, but I will keep these ideas in mind for a possible metrics standardization push.