Page MenuHomePhabricator

Donors to newcomers: analysis
Closed, ResolvedPublic

Description

On July 19, we will be sending emails to about 5,000 donors in Latin America asking them to create accounts and start editing. This task describes the analysis that we will want to do about three weeks after July 19. Three weeks gives recipients a week to take action on the email, and then two weeks for us to measure their behavior on wiki.

Key results

These are the most important measurements from this test. They should be broken down by platform (desktop/mobile) and wiki (Spanish/Portuguese). Both of these metrics will be generated by the Growth team.

  • Account creation rate: what percent of the donors we email create accounts? Our target is 5%.
    • The 5% was created by looking at the donation percent from the most recent LATAM email campaign, which was 10%. To make this KR, we halved that number to put it in a similar range, but to recognize that these people were more likely to donate (having been previous donors) than they are to make an account.
  • Constructive activation: what percent of the donor accounts make a first unreverted edit? Our target is TBD based on T286002: Donors to newcomers: baseline constructive activation.
    • We are going to set this key result using the baseline constructive activation rate in Spanish and Portuguese Wikipedias.
  • Clickthrough rate: though we will not be able to get these numbers directly from the emails, Growth team can approximate them through page loads of the landing pages. We'll want to know how many distinct IP addresses load the landing pages, broken down by wiki (Spanish/Portuguese) and platform (desktop/mobile).
Other measurements
  • Email interactions: these metrics are provided by Fundraising. We would like this broken down by wiki (Spanish/Portuguese), and by platform (desktop/mobile) and country if possible.
    • Open rate
    • Unsubscribe rate
    • Abuse rate
    • Bounce rate
    • Questions/replies: the number and general content of questions or replies received by Fundraising.
  • Account creation rate: what percent of recipients create accounts?
  • User interactions: we will compare these to baseline numbers for Spanish and Portuguese Wikipedias as set by organic account creations. We'll want these broken down by wiki (Spanish/Portuguese) and platform (desktop/mobile).
    • Activation: what percent of accounts make an article edit?
    • Constructive activation: what percent of accounts make an unreverted article edit?
    • Suggested edits activation: what percent of accounts make a suggested edit?
    • Mentorship: what percent of accounts ask a mentorship question?

Event Timeline

ldelench_wmf moved this task from Triage to Current Quarter on the Product-Analytics board.
nettrom_WMF closed this task as Resolved.EditedAug 10 2021, 11:06 PM

Darnit, commented on the wrong phab task, sorry. Nothing to see here, move along!

This work has been completed and the results shared with the Newcomer Pilot folks. We're planning to also share it on-wiki in the near future, but that's not currently on my plate.