@nettrom_WMF's initial statistics show that Vietnamese Wikipedia is having exceptionally high abandonment rates on the welcome survey for only desktop accounts.
- Starting on Jan 24, half of all new accounts have been receiving Variation A and half have been receiving Variation C of the welcome survey.
- We have been watching the results to make sure that the survey is not causing newcomers to leave Wikipedia right away. The percentage of newcomers who leave Wikipedia right away is called the "abandonment rate".
- We know from our experiments in Czech and Korean Wikipedias that having some abandonment is normal. Overall, abandonment in those wikis is about 10% when newcomers do not receive the survey -- in other words, 10% of newcomers leave the website immediately after creating their account, without doing anything else.
- We also saw in those experiments that receiving the survey did not increase that rate -- about the same number of newcomers left the website when they received the survey.
- But we're seeing something very different in Vietnamese Wikipedia. Among users who create their accounts on mobile, we see the same abandonment rates as in Czech and Korean: about 10%, for both Variation A and Variation C. But for users who create their accounts on desktop, we see very high abandonment rates: 46% for both Variation A and Variation C.
- If the survey is causing that many newcomers to abandon, then we need to change something very quickly. But first we want to do two things:
- Ask the community if they have any ideas about what could be going on.
- Change the experiment to include a control group so we can see what the normal abandonment rate is on desktop without the survey. We may start this next week.
- We have two hypotheses about what could be going on:
- There could be a real cultural difference between users who create accounts on mobile vs. those who create accounts on desktop. Perhaps desktop users are much less interested in surveys or do not want to answer questions.
- There could be something weird going on, in which many desktop accounts are not real accounts, perhaps created by vandals or bots. The reason we think this is possible is because of the clear difference between desktop and mobile.
Below is a table showing the differences in abandonment rate in Czech, Korean, and Vietnamese Wikipedias for Var A, Var C, and Control (with the exception of Vietnamese Wikipedia, which does not yet have a control group).
