The first pass of this analysis is now completed. In the original analysis in T228338 we only studied two Wikipedias, Czech and Korean, because those were the ones where the Homepage discovery features were deployed to. The new one has seven Wikipedias. We also want results split up by whether the user registered and/or visited on the desktop or mobile platform. This means we have 14 graphs. Instead of putting all of them in the same image, I chose to split this up by wiki.

I gathered data for all wikis for the first three months of 2020. The criteria were the same as in the previous analysis, meaning all registrations on the given wikis (thus no auto-created accounts), and excluding known test accounts, API registrations (these are mainly app accounts), users who don't have the Homepage enabled (we have 20% control groups on our four target wikis), and users who turned the Homepage on/off (self-selection). Data only counts the first visit a user makes to the Homepage, and we give every user a 48 hour window after registration to make their first visit. This also means that the last date of data is 2020-03-29.

Before making any plots of tables, I investigated what proportion of users make their first visit to the Homepage on a different platform than the one they registered on. If this proportion is very low, it would mean that we can exclude these users and thus reduce the number of categories we deal with. Across the dataset, 1.3% of users (314 out of 23,962) made that switch. I found this to be low enough that they could be excluded.

Visit Proportion Across Time:

Similarly as in the previous analysis, I made graphs for the proportion of registrations who visited the Homepage counted on a daily basis, split by platform. The date is the date of registration. A 7-day moving average is added to smooth out the variation. Wikis are listed in order of their database code (e.g. Armenian is fourth because the code is "hywiki"), and the same order is used in the subsequent tables below.

Arabic:

Czech:

Hungarian:

Armenian:

Korean:

Ukrainian:

Vietnamese:

Monthly overall proportion:

Next, I calculated the overall proportion for February and March, split by wiki and platform. I chose to hold January out because we deployed to three additional wikis in that month, or in other words, I only used months where we have (almost) complete data. In this case, The proportions were largely similar for both February and March, and I'm only reporting the March numbers because that month is the most recent. This results in the following table:

Routes to the Homepage:

Lastly, we look at the routes users took to get to the Homepage, based on the referer_route field in the HomepageVisit schema. In this case we also want to split the data by platform, but choose to use the is_mobile field in the schema rather than where the user registered so we can use all available data. In other words, the 1.3% of users who switch platforms are included. The (rather large) table below shows these proportions for the month of March:

While the task description doesn't ask questions, I think one of the patterns here is that we get a lot of visits through the Welcome Survey on the wikis where it's deployed. Thus we might want to consider whether the survey should be regarded as a key element in the newcomer experience.

Moving the task over to @MMiller_WMF's column for review.