Page MenuHomePhabricator

Donors to Newcomers: measure the success of test batch emails
Closed, ResolvedPublic

Description

A test batch of 200 emails will go out on July 14th. Before making a decision on whether to proceed with sending out the full set of emails, we want to measure whether any users signed up through this process. @nettrom_WMF will grab statistics on that and report back to the team.

We would like to check these numbers split by language wiki (Spanish and Portuguese).

Details

Due Date
Jul 15 2021, 2:00 PM

Event Timeline

nettrom_WMF set Due Date to Jul 15 2021, 2:00 PM.

This work has now been completed. The query that was run is below, which for simplicity doesn't exclude known test accounts. I also ran a query to manually inspect each account and removed one test account from the results. We then got the following table with number of signups:

wikinum_signups
ptwiki1
eswiki1

Query:

SELECT
    wiki,
    count(1) AS num_signups
FROM event.serversideaccountcreation
WHERE year = 2021
AND month = 7
AND ((day = 14 AND hour >= 19)
     OR day >= 15)
AND wiki IN ("ptwiki", "eswiki")
AND event.campaign REGEXP "^growth-advancement-"
GROUP BY wiki

I guess it might not be possible if there was only one signup per wiki, but it would be nice to see a mobile / desktop split too, as the UI is quite different so it's theoretically possible we only broke one of them.

I guess it might not be possible if there was only one signup per wiki, but it would be nice to see a mobile / desktop split too, as the UI is quite different so it's theoretically possible we only broke one of them.

I have a debug query that shows me whether the user signed up on desktop or mobile and on Portuguese there's now one of each.

We've done this and decided to proceed with the full mailing on July 19. The next set of analysis will take place in T286796: Donors to newcomers: analysis.