Page MenuHomePhabricator

Conduct analysis for Donation Prominence (WE 3.2.3) 15 days after deployment
Closed, ResolvedPublic

Description

Background
  • Beginning of phased release: October 8, 2024
  • Release date to 100%: October 10, 2024
  • 15 Days - October 25, 2024
The Task
  • Compare results to baseline data
  • Visualize and present the data in a way that is easily understandable to the team
At 15 days

Check:

  • Did we see a 50% increase in app menu donations as measured by pre-post analysis?
  • Pre-post analysis should include
    • Breakdown by day of App Menu donations for all countries together: for 15 days before beginning of phased release and 15 days after release to 100%
  • YoY Campaign country:
    • Breakdown by day of App Menu donations for Netherlands: for 15 days before beginning of phased release and 15 days after release to 100%
    • Breakdown by day of App Menu donations for Japan: for 15 days before beginning of phased release and 15 days after release to 100%
    • Breakdown by day of App Menu donations for France: for 15 days before beginning of phased release and 15 days after release to 100% (France has active campaign)
  • Deliver raw dataset containing all App Menu donations for all countries for both Native and Web payments for the experiment to the Hypothesis owner. Columns should include donation origin identifiers, country, language, payment method

Nice-to-have (if needed)

  • From Nov 1 - Dec 31, in Big English countries, what % increase did we see in App Menu donations in 2024 vs 2023
Important dates:
  • India App banners in August/September 2023
  • Banner features like "Maybe later", and "I already donated" added in October 2023
  • Native Apple Pay Release to production, October 24, 2023
  • Italy and Netherlands Campaigns in October/November 2023 (2023-10-18 - 2023-11-21)
  • Big English App Banners December 2023
  • Japan and Spain campaigns in June 2024
  • Upcoming App Banner campaigns
    • France (en, fr) App banners October 3 - 31, 2024
    • Netherlands (en, nl) App banners October 29 - November 26, 2024

Event Timeline

HNordeenWMF moved this task from Needs Triage to Up next on the Wikipedia-iOS-App-Backlog board.
HNordeenWMF updated the task description. (Show Details)
HNordeenWMF updated the task description. (Show Details)

This is done (except for last Nice-to-have), will share deck and spreadsheet on Slack.

Note: The raw data requested does not include the additional Payment Method dimension as that data is not accessible to me from Donations Dashboard. The Payment Method breakdown by each requested country is included in each specific country data sheet. If we want those extra values for all donations we will need to submit a request to FR Tech (via Asana) for date range 2024-09-22 - 2024-10-25.

To clarify for Baseline data I used the pre-release 15 day period for each country requested, were we thinking of using previous campaign dates by country as baseline instead?