INSTRUMENTATION SPECIFICATION: Instrumentation specs for DE1.3.1

Testable hypothesis

If we reduce cognitive load and provide more personalized post-account creation experiences for new account holders, then a greater proportion of users will constructively activate by making their first edit.

Experiment details

Overview

• Experiment Name: DE1.3.1 Early onboarding
• Tracking Key: DE1.3.1-early-onboarding
• Risk Level: [TBD] [according to Data Collection Guidelines, depending on data you plan to collect, including experiment-wide contextual attributes (see last section)]

[For Medium and High Risk only: Link to approved L3SC request in Asana]

Traffic

• Assign Variation by Attribute: MW User ID [survey is shown post-account-creation, so all subjects are logged in]
• Traffic Percent: 100%
• Variations: T430418: Account setup: Designs for early onboarding (DE1.3.1)
  • Control (control): Current Welcome Survey
  • Treatment (treatment): Redesigned account set-up experience.

• Targeting
  • MediaWiki Database Code in:
    • enwiki
    • frwiki
    • arwiki
    • eswiki

• Expected start date: [TBD]

Metrics

[REMINDER: The linked specification at the top should list all these metrics and their events, and which (if any) contextual attributes are needed for specific events.]

• Goal metrics
  • Constructive Activation

• Secondary metrics
  • Account set-up completion rate
  • Completion rate for each account set up question
  • Second-week retention (DE1.3 KR metric)
  • Proportion of accounts with an email

• Guardrail metrics
  • [TBD]