Page MenuHomePhabricator

Conduct analysis for iOS Temporary Accounts changes 30 days after deployment
Closed, ResolvedPublic

Description

Background

Before any Temporary Accounts deployments, both Android and Mobile web showed a prompt when a logged-out user started editing, asking them if they would like to create an account. The iOS has never shown a log-in prompt for when a logged-out editor begins to edit. The iOS app lets logged-out users begin editing as anonymous users right away. Along with other changes to the UI to support temporary accounts, we’ll be adding a full-screen prompt to log-in, and we want to measure how this is affecting our account creation and activation numbers

Updated user flow (Figma)NEW Full screen alert shown to logged-out users (Figma)
Screenshot 2025-03-04 at 9.34.06 AM.png (682×1 px, 70 KB)
image.png (1×750 px, 88 KB)
At 30 days after deployment of iOS UI changes:

For the Wikis where temporary accounts have been deployed,

  • Compare account creations in the 30 days after the Temporary Accounts UI deployment to the baseline. Did we see a relative increase?
  • Compare the constructive activation rate for the iOS app in the 30 days after the Temporary Accounts deployment to the baseline

Event Timeline

HNordeenWMF moved this task from Needs Triage to Up next on the Wikipedia-iOS-App-Backlog board.

Analysis feedback for @HNordeenWMF :

For tracking is_temp login and account create events we can use schema ios_login_action using the following columns/values that are already in use:

action = login_success, login_start, createaccount_start, impression, createaccount_success, logout

is_temp = for new temp accounts this value would be true throughout the login and create account process

We also have a category column that we could use if we wanted to add a value for temp accounts if needed (does not seem neceassry at this point).

If the question we want to answer is just 'Did we see a relative increase?' in new accounts the above tracking is nice to have but not necessary ( I assume it will be in effect by default by nature of how temp accounts are tracked).

@SNowick_WMF It would be nice to know that new accounts created were specifically temp accounts converting to real accounts.

If that is not already instrumented, we can fallback to looking for a relative increase in the languages where temp accounts is deployed.

Some questions are still being worked on

Constructive Activation result: Data

Date range% Activation% Increase
Pre-release 2025-04-01 - 2025-04-221.49%
Post-release 2025-04-23 - 2025-05-141.75%17.4%

@HNordeenWMF the data we use to get counts of new accounts does not track is_temp new accounts, once a user elects to create a Temp Account they drop out of our tracking (this is by design - see https://phabricator.wikimedia.org/T386532#10898696,).

I did track all new accounts created in Test Wikis Pre and Post Release,

Serverside Account New Accounts Temp-Enabled Wikis Only

Date RangeAvg Daily New Accounts% Increase
Pre-Release1.5
Post-release1.818.8%

Results Data

HNordeenWMF claimed this task.
HNordeenWMF added a subscriber: ppelberg.

Thanks @SNowick_WMF !

@ppelberg and @Niharika ! FYI that iOS added an account creation prompt as part of Temp accounts UI, that intercepted logged-out folks as they clicked "edit". (We didn't have that previously)
On pilot wikis, this resulted in a 17% increase in constructive activation on iOS (not signifcant though), and an 19% increase in average new (full) accounts created daily on the iOS App (from 1.5 to 1.8 Avg Daily New Accounts)