Page MenuHomePhabricator

Analyze welcome survey experiment in Arabic Wikipedia
Closed, ResolvedPublic

Description

The welcome survey was deployed to Arabic Wikipedia on 2019-07-15 with an experiment in which half of users receive Variation A and half receive no survey. After gathering about a month of data, we will analyze the differences in abandonment rate and differences in activation rate between the treatment and control groups.

This analysis should cover the same metrics as were pursued for other target wikis in T206380.

Event Timeline

kostajh moved this task from Inbox to Upcoming Work on the Growth-Team board.Jul 18 2019, 12:55 AM
nettrom_WMF triaged this task as Normal priority.Jul 24 2019, 5:38 PM
nettrom_WMF added a project: Product-Analytics.
nettrom_WMF moved this task from Triage to Next Up on the Product-Analytics board.

@nettrom_WMF has sent me the analysis on this, in which it looks like the Arabic abandonment and activation rates are not affected by the survey! That's good news. We're currently deciding where/when to publish the results.

nettrom_WMF closed this task as Resolved.Fri, Aug 30, 5:29 PM

The results have now been published: https://www.mediawiki.org/wiki/Growth/Analytics_updates/Welcome_survey_editor_activation_rate#Addendum%3A_Arabic_Wikipedia

One part of our analysis that is not part of the report is site abandonment. In Arabic Wikipedia, we did not find a significant increase in site abandonment. Rather, we found a slightly lower (but perhaps not significant) abandonment rate in the survey group, both on desktop and mobile.