Page MenuHomePhabricator
Create Task
Maniphest T429110

Temp Accounts A/B Test: Post-Publish Account Creation bottom sheet
Closed, ResolvedPublic5 Estimated Story Points
Actions

Description

As a follow up to T416418: [EPIC] Explore opportunities to guide Temporary Account users toward full account creation, configure and run an A/B test to measure the impact of the post-publish account creation bottom sheet on permanent account creation rates for Temporary Account users on mobile web.

Acceptance Criteria

  • Instrumentation is implemented to track exposure, CTA clicks, and account creation outcomes.
  • Primary metric: permanent account creation rate among Temporary Account users who publish an edit.
  • Experiment is configured with 50% Control (current experience) and 50% Treatment (details:T424205) variants.
  • Experiment is released to pilot wikis: French Wikipedia, Arabic Wikipedia, and Spanish Wikipedia.

QA Note:

  • To opt into the treatment group, run mw.testKitchen.overrideExperimentGroup( 'we-1-8-tempuser-post-edit', 'treatment') in the browser console.
  • Experiment monitoring confirms data is being collected correctly on all pilot wikis.

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
maybeSendThankYouEdit: avoid sending notification to temp usersMichael Großemediawiki/extensions/Echowmf/1.47.0-wmf.8+7 -40
postEdit: temp account experiment instrumentationMichael Großemediawiki/corewmf/1.47.0-wmf.8+59 -27
postEdit: temp account experiment instrumentationSergio Gimenomediawiki/coremaster+59 -27
maybeSendThankYouEdit: avoid sending notification to temp usersSergio Gimenomediawiki/extensions/Echomaster+7 -40
accountCreation: register temp account post edit experimentSergio Gimenomediawiki/extensions/WikimediaEventsmaster+3 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

KStoller-WMF created this task.Sun, Jun 14, 10:20 PM
Restricted Application added a project: Product Safety and Integrity. · View Herald TranscriptSun, Jun 14, 10:20 PM
Restricted Application added a subscriber: hubaishan. · View Herald Transcript
KStoller-WMF moved this task from Inbox to Needs Refinement on the Growth-Team board.Sun, Jun 14, 10:20 PM
Restricted Application added a subscriber: alaa. · View Herald TranscriptSun, Jun 14, 10:20 PM
KStoller-WMF renamed this task from A/B Test: Post-Publish Account Creation Bottom Sheet to Temp Accounts A/B Test: Post-Publish Account Creation bottom sheet.Sun, Jun 14, 10:22 PM
KStoller-WMF triaged this task as High priority.
Michael updated the task description. (Show Details)Mon, Jun 15, 1:28 PM
KStoller-WMF moved this task from Needs Refinement to Needs Estimation on the Growth-Team board.Mon, Jun 15, 1:58 PM
Sgs added a comment.EditedMon, Jun 15, 3:35 PM

Following discussion from T424205: Post-publish account creation nudge for temp accounts, it seems supporting an NCS edge-unique setup we require some extra work:

  1. Per T424205#12018085, we would need to somehow schedule the Echo notification from the client instead of the server. This different from how it is currently implemented which is via page updated event. Maybe Echo already has some API to schedule notifications client side? Alternatively @KStoller-WMF confirmed in T424205#12012514 it would be ok to not schedule these notifications at all for temporary accounts.
  2. Second, we would need to annotate the experiment name and group as we did for the account form improvements II in the redirects happening during to temporary account creation. @Michael rightly pointed that we only need the experiment and group where we record events so there's no need for ad-hoc annotation of these.
KStoller-WMF set the point value for this task to 5.Mon, Jun 15, 3:41 PM
KStoller-WMF moved this task from Needs Estimation to Up next Sprint on the Growth-Team board.
KStoller-WMF moved this task from Up next Sprint to FY2025-26 Q4 Sprint 5 on the Growth-Team board.Tue, Jun 16, 4:49 PM
KStoller-WMF edited projects, added: Growth-Team (FY2025-26 Q4 Sprint 5); removed: Growth-Team.
gerritbot added a comment.Thu, Jun 18, 4:00 PM

Change #1304108 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/core@master] postEdit: temp account experiment instrumentation

https://gerrit.wikimedia.org/r/1304108

gerritbot added a project: Patch-For-Review.Thu, Jun 18, 4:00 PM
gerritbot added a comment.Fri, Jun 19, 10:59 AM

Change #1304535 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/WikimediaEvents@master] accountCreation: register temp account post edit experiment

https://gerrit.wikimedia.org/r/1304535

gerritbot added a comment.Fri, Jun 19, 1:36 PM

Change #1304580 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/Echo@master] maybeSendThankYouEdit: avoid sending notification to temp users

https://gerrit.wikimedia.org/r/1304580

gerritbot added a comment.Mon, Jun 22, 11:26 AM

Change #1304535 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] accountCreation: register temp account post edit experiment

https://gerrit.wikimedia.org/r/1304535

ReleaseTaggerBot added a project: MW-1.47-notes (1.47.0-wmf.8; 2026-06-23).Mon, Jun 22, 12:00 PM
gerritbot added a comment.Tue, Jun 23, 9:30 AM

Change #1304580 merged by jenkins-bot:

[mediawiki/extensions/Echo@master] maybeSendThankYouEdit: avoid sending notification to temp users

https://gerrit.wikimedia.org/r/1304580

gerritbot added a comment.Tue, Jun 23, 12:25 PM

Change #1304108 merged by jenkins-bot:

[mediawiki/core@master] postEdit: temp account experiment instrumentation

https://gerrit.wikimedia.org/r/1304108

Maintenance_bot removed a project: Patch-For-Review.Tue, Jun 23, 12:31 PM
ReleaseTaggerBot edited projects, added: MW-1.47-notes (1.47.0-wmf.9; 2026-06-30); removed: MW-1.47-notes (1.47.0-wmf.8; 2026-06-23).Tue, Jun 23, 1:00 PM
Sgs added a comment.Tue, Jun 23, 1:50 PM

Summary report:

EMcFarland-WMF updated the task description. (Show Details)Tue, Jun 23, 3:09 PM
DMburugu updated the task description. (Show Details)Tue, Jun 23, 3:21 PM
DMburugu added a project: OKR-Work.Wed, Jun 24, 12:05 PM
Michael claimed this task.Mon, Jun 29, 1:44 PM
Michael moved this task from Incoming to Doing on the Growth-Team (FY2025-26 Q4 Sprint 5) board.

The plan is to turn this on today.

Restricted Application added a project: User-Michael. · View Herald TranscriptMon, Jun 29, 1:44 PM
gerritbot added a comment.Mon, Jun 29, 3:12 PM

Change #1306341 had a related patch set uploaded (by Michael Große; author: Sergio Gimeno):

[mediawiki/core@wmf/1.47.0-wmf.8] postEdit: temp account experiment instrumentation

https://gerrit.wikimedia.org/r/1306341

gerritbot added a project: Patch-For-Review.Mon, Jun 29, 3:12 PM

Change #1306342 had a related patch set uploaded (by Michael Große; author: Sergio Gimeno):

[mediawiki/extensions/Echo@wmf/1.47.0-wmf.8] maybeSendThankYouEdit: avoid sending notification to temp users

https://gerrit.wikimedia.org/r/1306342

KStoller-WMF added a comment.Mon, Jun 29, 9:19 PM

Experiment enabled via GrowthBook, but won't actually start until next TestKitchen deployment window (I believe Tuesday, June 30th ~15UTC).

https://growthbook.wikimedia.org/experiment/exp_10ipnmqzch3jt#results
https://test-kitchen.wikimedia.org/experiment/we-1-8-tempuser-post-edit

gerritbot added a comment.Tue, Jun 30, 1:22 PM

Change #1306341 merged by jenkins-bot:

[mediawiki/core@wmf/1.47.0-wmf.8] postEdit: temp account experiment instrumentation

https://gerrit.wikimedia.org/r/1306341

gerritbot added a comment.Tue, Jun 30, 1:22 PM

Change #1306342 merged by jenkins-bot:

[mediawiki/extensions/Echo@wmf/1.47.0-wmf.8] maybeSendThankYouEdit: avoid sending notification to temp users

https://gerrit.wikimedia.org/r/1306342

Stashbot mentioned this in T424205: Post-publish account creation nudge for temp accounts.Tue, Jun 30, 1:23 PM

Mentioned in SAL (#wikimedia-operations) [2026-06-30T13:23:13Z] <lucaswerkmeister-wmde@deploy1003> Started scap sync-world: Backport for [[gerrit:1306341|postEdit: temp account experiment instrumentation (T429110)]], [[gerrit:1306342|maybeSendThankYouEdit: avoid sending notification to temp users (T429110 T424205)]]

Stashbot added a comment.Tue, Jun 30, 1:25 PM

Mentioned in SAL (#wikimedia-operations) [2026-06-30T13:25:18Z] <lucaswerkmeister-wmde@deploy1003> lucaswerkmeister-wmde, migr: Backport for [[gerrit:1306341|postEdit: temp account experiment instrumentation (T429110)]], [[gerrit:1306342|maybeSendThankYouEdit: avoid sending notification to temp users (T429110 T424205)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Maintenance_bot removed a project: Patch-For-Review.Tue, Jun 30, 1:31 PM
Stashbot added a comment.Tue, Jun 30, 1:38 PM

Mentioned in SAL (#wikimedia-operations) [2026-06-30T13:38:25Z] <lucaswerkmeister-wmde@deploy1003> Finished scap sync-world: Backport for [[gerrit:1306341|postEdit: temp account experiment instrumentation (T429110)]], [[gerrit:1306342|maybeSendThankYouEdit: avoid sending notification to temp users (T429110 T424205)]] (duration: 15m 12s)

ReleaseTaggerBot edited projects, added: MW-1.47-notes (1.47.0-wmf.8; 2026-06-23); removed: MW-1.47-notes (1.47.0-wmf.9; 2026-06-30).Tue, Jun 30, 2:01 PM
Michael moved this task from Doing to QA on the Growth-Team (FY2025-26 Q4 Sprint 5) board.Tue, Jun 30, 2:47 PM

The relevant changes have been backported and the experiment has been enabled. I'm also seeing data coming in with the following kafkacat command on stat1011:

kafkacat -C -b kafka-jumbo1010.eqiad.wmnet:9092 \
  -t eqiad.product_metrics.web_base -o end | \
  jq 'select(.experiment.enrolled == "we-1-8-tempuser-post-edit")'

I've seen edit_saved, impression, account_created, and experiment_exposure events so far, thus I think things seem to be working correctly 👍.

DMburugu edited projects, added: Growth-Team (FY2026-27 Q1 Sprint 1); removed: Growth-Team (FY2025-26 Q4 Sprint 5).Tue, Jun 30, 4:35 PM
DMburugu moved this task from Incoming to QA on the Growth-Team (FY2026-27 Q1 Sprint 1) board.
DMburugu closed this task as Resolved.EditedWed, Jul 1, 2:50 PM

Since we've confirmed that data is coming in and results are showing up on Growthbook(requires authorisation to view), we'll consider the A/B test up and running so this can be resolved.

We are following up on a higher traffic allocation with the Experiment Platform team as well T430730: Retroactively configure Temp Accounts WE 1.8.6 experiment in TK UI to get a larger sample size for the editing action.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits