Page MenuHomePhabricator

hCaptcha: Enable A/B test for frwiki
Closed, ResolvedPublic

Description

Summary

We want to A/B test the hCaptcha-enabled Special:CreateAccount form on frwiki.

Technical notes

  • Set up an experiment in xLab using edge uniques
  • Set up a fallback in Hooks::getInstance() or in operations/mediawiki-config when deciding to load hCaptcha + related config.
  • Add a field / event in the special_create_account instrumentation to indicate that a particular funnel is loading FancyCaptcha or hCaptcha

Acceptance criteria

  • 90% of traffic on frwiki sees the hCaptcha-enabled Special:CreateAccount form, 10% sees FancyCaptcha
  • special_create_account instrument indicates if the user is interacting with FancyCaptcha or hCaptcha

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
operations/mediawiki-configmaster+1 -12
operations/mediawiki-configmaster+5 -0
mediawiki/extensions/WikimediaEventsmaster+109 -5
operations/mediawiki-configmaster+14 -0
mediawiki/extensions/WikimediaEventswmf/1.45.0-wmf.20+44 -8
mediawiki/extensions/WikimediaEventswmf/1.45.0-wmf.21+44 -8
mediawiki/extensions/WikimediaEventsmaster+44 -8
mediawiki/extensions/WikimediaEventsmaster+43 -35
mediawiki/extensions/WikimediaEventswmf/1.45.0-wmf.20+43 -35
mediawiki/extensions/WikimediaEventswmf/1.45.0-wmf.21+43 -35
mediawiki/extensions/WikimediaEventswmf/1.45.0-wmf.20+93 -32
mediawiki/extensions/WikimediaEventswmf/1.45.0-wmf.21+93 -32
mediawiki/extensions/WikimediaEventsmaster+93 -32
integration/configmaster+2 -0
mediawiki/extensions/ConfirmEditwmf/1.45.0-wmf.20+138 -6
mediawiki/extensions/ConfirmEditmaster+138 -6
Show related patches Customize query in gerrit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

My thought is to take the ConfirmEditGetHooksInstanceHook code from https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ConfirmEdit/+/1189821 and use it for this implementation.

Change #1190297 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/ConfirmEdit@master] Hooks: Enable overriding the hook instance

https://gerrit.wikimedia.org/r/1190297

Change #1190992 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/mediawiki-config@master] WIP hCaptcha: Define configuration for A/B test

https://gerrit.wikimedia.org/r/1190992

Change #1190297 merged by jenkins-bot:

[mediawiki/extensions/ConfirmEdit@master] Hooks: Enable overriding the hook instance per action

https://gerrit.wikimedia.org/r/1190297

Change #1192148 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/ConfirmEdit@wmf/1.45.0-wmf.20] Hooks: Enable overriding the hook instance per action

https://gerrit.wikimedia.org/r/1192148

Change #1192148 merged by jenkins-bot:

[mediawiki/extensions/ConfirmEdit@wmf/1.45.0-wmf.20] Hooks: Enable overriding the hook instance per action

https://gerrit.wikimedia.org/r/1192148

Mentioned in SAL (#wikimedia-operations) [2025-09-30T06:27:20Z] <kharlan@deploy2002> Started scap sync-world: Backport for [[gerrit:1192148|Hooks: Enable overriding the hook instance per action (T405239 T404204)]]

Mentioned in SAL (#wikimedia-operations) [2025-09-30T06:33:29Z] <kharlan@deploy2002> kharlan: Backport for [[gerrit:1192148|Hooks: Enable overriding the hook instance per action (T405239 T404204)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-09-30T06:42:29Z] <kharlan@deploy2002> Finished scap sync-world: Backport for [[gerrit:1192148|Hooks: Enable overriding the hook instance per action (T405239 T404204)]] (duration: 15m 09s)

Change #1192544 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/WikimediaEvents@master] CreateAccount: Record the CAPTCHA class used in the account creation

https://gerrit.wikimedia.org/r/1192544

Change #1192550 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/config@master] zuul: WikimediaEvents, define dependency on ConfirmEdit

https://gerrit.wikimedia.org/r/1192550

Change #1192550 merged by jenkins-bot:

[integration/config@master] Zuul: [mediawiki/extensions/WikimediaEvents] Add dependency on ConfirmEdit

https://gerrit.wikimedia.org/r/1192550

Mentioned in SAL (#wikimedia-releng) [2025-09-30T18:55:31Z] <James_F> Zuul: [mediawiki/extensions/WikimediaEvents] Add dependency on ConfirmEdit, for T405239

Change #1192544 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] CreateAccount: Record the CAPTCHA class used in account creation funnel

https://gerrit.wikimedia.org/r/1192544

Change #1192668 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/WikimediaEvents@wmf/1.45.0-wmf.20] CreateAccount: Record the CAPTCHA class used in account creation funnel

https://gerrit.wikimedia.org/r/1192668

Change #1192670 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/WikimediaEvents@wmf/1.45.0-wmf.21] CreateAccount: Record the CAPTCHA class used in account creation funnel

https://gerrit.wikimedia.org/r/1192670

Change #1192670 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@wmf/1.45.0-wmf.21] CreateAccount: Record the CAPTCHA class used in account creation funnel

https://gerrit.wikimedia.org/r/1192670

Mentioned in SAL (#wikimedia-operations) [2025-10-01T06:17:35Z] <kharlan@deploy2002> Started scap sync-world: Backport for [[gerrit:1192670|CreateAccount: Record the CAPTCHA class used in account creation funnel (T405239)]], [[gerrit:1192667|CreateAccount: Track interactions with the captchaWord field (T394744)]]

Mentioned in SAL (#wikimedia-operations) [2025-10-01T06:22:36Z] <kharlan@deploy2002> kharlan: Backport for [[gerrit:1192670|CreateAccount: Record the CAPTCHA class used in account creation funnel (T405239)]], [[gerrit:1192667|CreateAccount: Track interactions with the captchaWord field (T394744)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-10-01T06:40:09Z] <kharlan@deploy2002> Finished scap sync-world: Backport for [[gerrit:1192670|CreateAccount: Record the CAPTCHA class used in account creation funnel (T405239)]], [[gerrit:1192667|CreateAccount: Track interactions with the captchaWord field (T394744)]] (duration: 22m 34s)

Change #1192780 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/WikimediaEvents@master] CreateAccountInstrumentationPreAuthenticationProvider: Fix server side logging of CAPTCHA class

https://gerrit.wikimedia.org/r/1192780

Change #1192668 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@wmf/1.45.0-wmf.20] CreateAccount: Record the CAPTCHA class used in account creation funnel

https://gerrit.wikimedia.org/r/1192668

Mentioned in SAL (#wikimedia-operations) [2025-10-01T06:55:51Z] <kharlan@deploy2002> Started scap sync-world: Backport for [[gerrit:1192666|CreateAccount: Track interactions with the captchaWord field (T394744)]], [[gerrit:1192668|CreateAccount: Record the CAPTCHA class used in account creation funnel (T405239)]]

Mentioned in SAL (#wikimedia-operations) [2025-10-01T07:02:07Z] <kharlan@deploy2002> kharlan: Backport for [[gerrit:1192666|CreateAccount: Track interactions with the captchaWord field (T394744)]], [[gerrit:1192668|CreateAccount: Record the CAPTCHA class used in account creation funnel (T405239)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-10-01T07:10:00Z] <kharlan@deploy2002> Finished scap sync-world: Backport for [[gerrit:1192666|CreateAccount: Track interactions with the captchaWord field (T394744)]], [[gerrit:1192668|CreateAccount: Record the CAPTCHA class used in account creation funnel (T405239)]] (duration: 14m 09s)

Remaining tasks:

Change #1192780 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] CreateAccount: Fix server side logging of CAPTCHA class

https://gerrit.wikimedia.org/r/1192780

Change #1192852 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/WikimediaEvents@wmf/1.45.0-wmf.20] CreateAccount: Fix server side logging of CAPTCHA class

https://gerrit.wikimedia.org/r/1192852

Change #1192853 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/WikimediaEvents@wmf/1.45.0-wmf.21] CreateAccount: Fix server side logging of CAPTCHA class

https://gerrit.wikimedia.org/r/1192853

Change #1192853 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@wmf/1.45.0-wmf.21] CreateAccount: Fix server side logging of CAPTCHA class

https://gerrit.wikimedia.org/r/1192853

Change #1192852 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@wmf/1.45.0-wmf.20] CreateAccount: Fix server side logging of CAPTCHA class

https://gerrit.wikimedia.org/r/1192852

Mentioned in SAL (#wikimedia-operations) [2025-10-01T09:44:02Z] <kharlan@deploy2002> Started scap sync-world: Backport for [[gerrit:1192853|CreateAccount: Fix server side logging of CAPTCHA class (T405239)]], [[gerrit:1192852|CreateAccount: Fix server side logging of CAPTCHA class (T405239)]]

Mentioned in SAL (#wikimedia-operations) [2025-10-01T09:50:37Z] <kharlan@deploy2002> kharlan: Backport for [[gerrit:1192853|CreateAccount: Fix server side logging of CAPTCHA class (T405239)]], [[gerrit:1192852|CreateAccount: Fix server side logging of CAPTCHA class (T405239)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-10-01T09:59:49Z] <kharlan@deploy2002> Finished scap sync-world: Backport for [[gerrit:1192853|CreateAccount: Fix server side logging of CAPTCHA class (T405239)]], [[gerrit:1192852|CreateAccount: Fix server side logging of CAPTCHA class (T405239)]] (duration: 15m 47s)

Change #1192874 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/WikimediaEvents@master] WIP: Don't log if user can skip CAPTCHA

https://gerrit.wikimedia.org/r/1192874

Change #1192874 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] CreateAccountInstrumentationPreAuthenticationProvider: Don't create event if user can skip CAPTCHA

https://gerrit.wikimedia.org/r/1192874

Change #1192924 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/WikimediaEvents@wmf/1.45.0-wmf.20] CreateAccountInstrumentationPreAuthenticationProvider: Don't create event if user can skip CAPTCHA

https://gerrit.wikimedia.org/r/1192924

Change #1192925 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/WikimediaEvents@wmf/1.45.0-wmf.21] CreateAccountInstrumentationPreAuthenticationProvider: Don't create event if user can skip CAPTCHA

https://gerrit.wikimedia.org/r/1192925

Remaining tasks:

This is done

  • wait for the MPIC experiment to be started and for the end date to be extended to December 3, 2025

This is done

I'm backporting a one more minor instrumentation adjustment, then will enable the A/B test patch.

Change #1192925 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@wmf/1.45.0-wmf.21] CreateAccountInstrumentationPreAuthenticationProvider: Don't create event if user can skip CAPTCHA

https://gerrit.wikimedia.org/r/1192925

Change #1192924 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@wmf/1.45.0-wmf.20] CreateAccountInstrumentationPreAuthenticationProvider: Don't create event if user can skip CAPTCHA

https://gerrit.wikimedia.org/r/1192924

Mentioned in SAL (#wikimedia-operations) [2025-10-01T16:10:26Z] <kharlan@deploy2002> Started scap sync-world: Backport for [[gerrit:1192922|SimpleCaptcha::canSkipCaptcha: Remove unneeded Config parameter]], [[gerrit:1192924|CreateAccountInstrumentationPreAuthenticationProvider: Don't create event if user can skip CAPTCHA (T405239)]], [[gerrit:1192925|CreateAccountInstrumentationPreAuthenticationProvider: Don't create event if user can skip CAPTCHA (T405239)]], [[gerrit:1192923|Simp

Mentioned in SAL (#wikimedia-operations) [2025-10-01T16:17:07Z] <kharlan@deploy2002> kharlan: Backport for [[gerrit:1192922|SimpleCaptcha::canSkipCaptcha: Remove unneeded Config parameter]], [[gerrit:1192924|CreateAccountInstrumentationPreAuthenticationProvider: Don't create event if user can skip CAPTCHA (T405239)]], [[gerrit:1192925|CreateAccountInstrumentationPreAuthenticationProvider: Don't create event if user can skip CAPTCHA (T405239)]], [[gerrit:1192923|SimpleCaptcha::canSk

Mentioned in SAL (#wikimedia-operations) [2025-10-01T16:23:35Z] <kharlan@deploy2002> Finished scap sync-world: Backport for [[gerrit:1192922|SimpleCaptcha::canSkipCaptcha: Remove unneeded Config parameter]], [[gerrit:1192924|CreateAccountInstrumentationPreAuthenticationProvider: Don't create event if user can skip CAPTCHA (T405239)]], [[gerrit:1192925|CreateAccountInstrumentationPreAuthenticationProvider: Don't create event if user can skip CAPTCHA (T405239)]], [[gerrit:1192923|Sim

Change #1190992 merged by jenkins-bot:

[operations/mediawiki-config@master] hCaptcha: Enable A/B test for frwiki

https://gerrit.wikimedia.org/r/1190992

Mentioned in SAL (#wikimedia-operations) [2025-10-01T18:46:45Z] <kharlan@deploy2002> Started scap sync-world: Backport for [[gerrit:1190992|hCaptcha: Enable A/B test for frwiki (T405239)]]

Mentioned in SAL (#wikimedia-operations) [2025-10-01T18:53:03Z] <kharlan@deploy2002> kharlan: Backport for [[gerrit:1190992|hCaptcha: Enable A/B test for frwiki (T405239)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-10-01T19:13:09Z] <kharlan@deploy2002> Finished scap sync-world: Backport for [[gerrit:1190992|hCaptcha: Enable A/B test for frwiki (T405239)]] (duration: 26m 24s)

Change #1193071 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/WikimediaEvents@master] frwiki hCaptcha A/B test: Preserve experiment group in query

https://gerrit.wikimedia.org/r/1193071

Change #1193096 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/mediawiki-config@master] WikimediaEventsAuthPreserveQueryParamsExperiments: Define hCaptcha A/B test

https://gerrit.wikimedia.org/r/1193096

Change #1193071 abandoned by Kosta Harlan:

[mediawiki/extensions/WikimediaEvents@master] Implement AuthPreserveQueryParams for Metrics Platform mpo param

Reason:

See I579f168071cf03adac988f3863dcb56a2f40237c instead

https://gerrit.wikimedia.org/r/1193071

Change #1193096 merged by jenkins-bot:

[operations/mediawiki-config@master] MetricsPlatformAuthPreserveQueryParamsExperiments: Define hCaptcha A/B test

https://gerrit.wikimedia.org/r/1193096

Mentioned in SAL (#wikimedia-operations) [2025-10-06T07:37:34Z] <kharlan@deploy2002> Started scap sync-world: Backport for [[gerrit:1193096|MetricsPlatformAuthPreserveQueryParamsExperiments: Define hCaptcha A/B test (T405239)]]

Mentioned in SAL (#wikimedia-operations) [2025-10-06T07:43:54Z] <kharlan@deploy2002> kharlan: Backport for [[gerrit:1193096|MetricsPlatformAuthPreserveQueryParamsExperiments: Define hCaptcha A/B test (T405239)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-10-06T07:49:16Z] <kharlan@deploy2002> Finished scap sync-world: Backport for [[gerrit:1193096|MetricsPlatformAuthPreserveQueryParamsExperiments: Define hCaptcha A/B test (T405239)]] (duration: 11m 42s)

kostajh updated the task description. (Show Details)

Change #1218780 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/mediawiki-config@master] hCaptcha: end frwiki A/B test

https://gerrit.wikimedia.org/r/1218780

Change #1218780 merged by jenkins-bot:

[operations/mediawiki-config@master] hCaptcha: end frwiki A/B test

https://gerrit.wikimedia.org/r/1218780

Mentioned in SAL (#wikimedia-operations) [2025-12-16T15:16:34Z] <kharlan@deploy2002> Started scap sync-world: Backport for [[gerrit:1218780|hCaptcha: end frwiki A/B test (T405239)]]

Mentioned in SAL (#wikimedia-operations) [2025-12-16T15:18:54Z] <kharlan@deploy2002> kharlan: Backport for [[gerrit:1218780|hCaptcha: end frwiki A/B test (T405239)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-12-16T15:30:01Z] <kharlan@deploy2002> Finished scap sync-world: Backport for [[gerrit:1218780|hCaptcha: end frwiki A/B test (T405239)]] (duration: 13m 26s)