Page MenuHomePhabricator

Update Measurement Plan and Instrument GPay
Open, MediumPublic

Description

Background

We are adding native GPay into the Android App so that donors can give without leaving the App. We plan to measure its impact on overall donation completion rate, and verify if it is providing a more streamlined donation experience in the App.

Tasks
  • Augment Android measurement plan and instrumentation docs to include Android GPay donor events based on screens from Figma / Design ticket: T362698
  • Instrument GPay based on Doc from Shay, ensure it is released along with first rollout to Japan
  • Create data validation task
How we'll measure success

Plan to report on specific metrics for GPay:

  • Indicator: 30% higher donation completion rate (Success rate %) for donations made through Native GPay as compared to web view donations (all payment types)
  • Indicator: 25% increase in donation completion rate (Success rate %) for all donations made in Android app during campaign as compared to previous year (Use Japan Campaign from this year & last year)
  • Curiosity: What percentage of users use native payment GPay compared to GPay on Web?
  • Curiosity: How did the share of GPay donations as a portion of all Android donations change? (compare to last campaign, or before and after of Apps Menu donations)
  • Guardrail: Abandonment rate is no more than 25%

Event Timeline

HNordeenWMF renamed this task from Augment Android measurement plan and instrumentation docs to include Android GPay donor events to Update Measurement Plan and Instrument GPay.Tue, Apr 30, 5:15 PM
HNordeenWMF updated the task description. (Show Details)

Instrumentation for GPay has been added to main instrumentation doc Slides 18-22. Note: DB had suggested we use the breadcrumbs table for this but doing so would put all our event data for this one engagement into a separate table, similar events already exist for iOS in the app_donor_experience dataset which we will need to compare so it just makes sense to keep them all in one place.

HNordeenWMF subscribed.

Thanks @SNowick_WMF! This task can be repurposed for engineers to implement the GPay instrumentation above.

HNordeenWMF updated the task description. (Show Details)

@SNowick_WMF will you be using the metrics platform data contract for this measurement? Or are you repurposing an existing instrument?

HI @VirginiaPoundstone the rest of these events are already instrumented on MEP so we will be using that schema (app_donor_experience). We are adding a new user payment method to already existing user flows.