Page MenuHomePhabricator

Determine how constructive activation has evolved/fluctuated over time
Closed, ResolvedPublic

Description

This ticket is meant to help us understand how – if at all – the constructive activation rate has evolved over time.

Knowing this, will help us decide the extent to which we need to account for historical/cyclical fluctuations in the target improvement we're setting in T371726.

Note: work on this task will happen after T371737.

Reason being: we expect the past analyses we'll be reviewing in T371737 to – in part – provide the data needed to determine the presence (or lack of!) the fluctuations this task is asking us to identify.

Event Timeline

ppelberg renamed this task from Estimate magnitude of y-o-y change we can cause in constructive activation to Determine how constructive activation has evolved/fluctuated over time.Aug 7 2024, 8:56 PM
ppelberg updated the task description. (Show Details)

I reviewed constructive activation rates across recent registration months as well as YoY trends to confirm how this metric as defined [i] fluctuates over time. See the findings documented below.

As it's been a few months since the baseline calculation was completed in T360829, I reran the analysis to obtain May and June 2024 constructive activation rates. See per registration month activation rates from March to June 2024 and year-over-year (YoY) relative changes:

Mobile Web Constructive Activation Rates by Registration Month

Registration MonthNumber of accounts createdNumber of Users ActivatedProportion of UsersYoY Relative Change
March 2024701141254417.9%+ 10.2%
April 2024703251219317.3%+ 8.0
May 2024716221221517.1%+ 5.4%
June 2024612171144818.7%+ 8.8%

Average constructive activation rate of 17.7% on mobile web.

Mobile Web Registration month changes
Per month activation rates for mobile web have been fairly stable since March 2024, fluctuating between 17 and 18%. The highest month over month increase was observed from May to June 2024, where rates increased from 17.0% to 18.7% . This may be partly due to a decrease in account creations between May and June likely due to seasonal changes associated with the end of the school year and vacation.

Mobile Web YoY changes
Constructive activation rates have consistently increased YoY from 2023-2024 for the reviewed months (March-June). These increases have ranged from a +10.2% in March 2024 to +5.4% in May 2024.

Mobile App Constructive Activation Rates by Registration Month

Registration MonthNumber of accounts createdNumber of Users ActivatedProportion of UsersYoY Relative Change
March 2024196814302.2%+ 38.4%
April 2024184123872.1%+23.5%
May 2024180243552.0%+ 18.3%
June 2024170173642.1%+ 9.4%

Average constructive activation rate of 2.1%. on mobile apps

Mobile App Registration month changes
Mobile app constructive activation rates have been stable since March 2024, fluctuating only ±0.1 percentage points each month.

Mobile App Registration YoY changes
However, mobile app constructive activation rates have increased slightly year over year compared to 2023.

Note: There is more fluctuation in relative YoY changes for mobile apps because the activation rate is low compared to mobile web. For example, in March 2024, there was a 2.2% constructive activation rate on mobile apps compared to 1.59% in March 2023. This is a 38.4% increase but only a 0.6 percentage point absolute increase. Note: While there was also a decrease in mobile app account registrations between May and June 2024, it did not appear to significantly impact the constructive activation rate in this case.

YoY June trends 2019-2024

Below rates logged for June from 2019 through 2024 to help identify any longstanding YoY or seasonal trends .

June YOY Changes in Mobile Web Constructive Activation Rates

Registration Month and YearNumber of accounts createdNumber of Users ActivatedProportion of UsersRelative YoY ChangeAbsolute YoY Change
June 2021836581304315.6%8.2%1.2 pp
June 2022902891414715.7%0.5%0.08 pp
June 2023786051350217.2%9.6%1.5 pp
June 2024612171144818.7%8.8%1.5 pp

Mobile web constructive activation rates have been increasing year over year since 2021, with YoY increases ranging from 0.5 to 9.6% .

June YOY Changes in Mobile Apps Constructive Activation Rates

Registration Month and YearNumber of accounts createdNumber of Users ActivatedProportion of UsersRelative YoY ChangeAbsolute YoY Change
June 2021206094422.1%174.4%1.4 pp
June 2022190483321.7%-18.7%-0.4 pp
June 2023171033281.9%10.3%0.2 pp
June 2024170173642.1%9.4%0.2 pp

Mobile app constructive activation rates have fluctuated up and down year over year but there have been no significant changes in activation rates since 2021.

Data also indicates that the number of new accounts created on both mobile apps and web in June have been steadily decreasing since 2021 but there is not a similar decrease in activated users. As a result, we've still observed some increases in activation rates.


i. Where "constructive activation" refers to the percentage of brand new editors who publish ≥1 edit in the main namespace on a mobile device that is not reverted within 48 hours, as measured by controlled experiments.