Page MenuHomePhabricator

Establish baselines for account creation behaviour
Open, Needs TriagePublic

Description

Now that Special:CreateAccount is instrumented, we are looking to analyse the data to establish baselines for account creation behaviour in preparation for the hCaptcha trial. In this analysis, we are looking to answer the following questions:

  1. What percentage of users who start filling out the form do not complete it?
  2. How long does it typically take users to submit the form? We'll report median, 75th, 90th, 95th, and 99th percentiles, before considering reporting it based on the number of seconds.
  3. What percentage of users are submitting the form without JavaScript enabled?
  4. What proportion of submissions results in a CAPTCHA failure?
  5. What proportion of submissions that result in a CAPTCHA failure is followed by a CAPTCHA success?
  6. At what step in the account creation funnel do users who start filling out the form bounce?

Event Timeline

I used queries that we developed for the hCaptcha account creation A/B test analysis for a snapshot analysis, with data from the first three weeks of January 2026. We find that the data allows us to easily answer the first two questions, and that we're unfortunately unable to answer the last question, which might be the most important one to answer. I'll be filing a subtask of T394744 to update the instrumentation to fix that problem.

What percentage of users who start filling out the form do not complete it?

This is called the abandonment rate, and it's the converse of the completion rate. For wikis that do not have hCaptcha enabled (all but eight wikis), we get the following table:

PlatformN funnel startedN funnel completedCompletion rateAbandonment rate
Desktop web183.4k29.4k16.0%84.0%
Mobile web62.7k20.0k31.8%68.2%
Total246.1k49.4k20.1%79.9%

The same table for the eight wikis that have hCaptcha enabled for account creation (Chinese, English, Indonesian, Japanese, French, Persian, Portuguese, and Turkish):

PlatformN funnel startedN funnel completedCompletion rateAbandonment rate
Desktop web241.3k42.5k17.6%82.4%
Mobile web95.3k25.0k26.2%73.8%
Total336.6k67.5k20.0%80.0%

How long does it typically take users to submit the form?

For both categories of wikis (hCaptcha and non-hCaptcha as defined above) and across both platforms, the median time to completion is less than a minute.

A place where we find some differences is that the eight wikis that have hCaptcha see a higher proportion of account creations that are completed relatively quickly. For the "less than 30 seconds" category, hCaptcha wikis have a substantially higher proportion (desktop web: 26.9%, mobile web: 24.6%), than the other wikis (desktop web: 19.3%, mobile web: 16.1%). Additional analysis is needed to understand more about these behavioural differences, e.g. if they're driven by automated creations or if there are other reasons behind them).

Additional observations:
We've previously pulled data twice and see a lot of variation in completion rates. This indicates that we might be interested in monitoring completion rates more closely, i.e setting up an Airflow job to aggregate it on a daily basis. That would then allow us to possibly identify challenges such as significant increases in creation attempts at specific wikis, sudden changes in completion rate, etc.