Page MenuHomePhabricator

[WE 1.2] Generate snapshot of construction activation fluctuation since June 2024
Closed, ResolvedPublic

Description

As we near the end of the 2024-2025 fiscal year on June 30, 2025, we would like to use this task to evaluate what – if any – year-over-year changes we observe in the constructive activation rates on mobile web and iOS.

Timing

We'll begin this analysis the week of July 9th. By this time, MediaWiki Snapshot data will be available for June 2025.

Requirements

pull from T360829

Done

Event Timeline

MNeisler triaged this task as Medium priority.Jul 2 2025, 2:53 PM
MNeisler moved this task from Triage to Current Quarter on the Product-Analytics board.

Below are the recent constructive activation rates for June 2025:

Mobile Web: 17.7% ( No YoY change based on 3-month rolling average )
While we observed a decrease in Mobile Web constructive activation rates for June 2025, this is partly because we observed a spike in rates last June 2024 to 18.7%, which was not sustained. Using the 3-month rolling average to smooth out the influnce of specific events, there was no YoY change in mobile web constructive activation rates from June 2024 to June 2025.

Mobile Apps: 2.5% (+23% YoY change based on 3-month rolling average)

  • iOS: 1.8%
  • Android: 2.8%

Note: Year-over-year data not available for iOS and Android breakdowns as this data is scrubbed after 90 days.
Mobile app rates continue to average around 2%; however, we have observed YoY increases. Constructive activation rates increased from 2.2% in June 2024 to 2.5% in June 2025.

Here is the updated slide deck where you can view updated trend line charts showing monthly constructive activations have fluctuated from March 2024 through June 2025.

cc @ppelberg

Below are the recent constructive activation rates for June 2025:\

@MNeisler thank you for bring this all together...

A few clarifying questions for you upon revisiting these findings after what we talked about offline earlier today...

Mobile Web: 17.7% ( No YoY change based on 3-month rolling average )
...Using the 3-month rolling average to smooth out the influnce of specific events, there was no YoY change in mobile web constructive activation rates from June 2024 to June 2025.

Question #1: What – if anything – about the quoted bit below would need to be changed to accurately describe the approach you described above?

"To smooth out the influence of specific events, this analysis calculates YoY changes in mobile web constructive activation rates by comparing the average 3-month rolling CA rate in June 2024 (April, May, June 2024) to the average 3-month rolling CA rate in June 2025 (April, May, June 2025). When comparing these 2024 and 2025 rolling averages, we observe no change in the constructive activation rate."

Mobile Apps: 2.5% (+23% YoY change based on 3-month rolling average)

  • iOS: 1.8%
  • Android: 2.8%

Question #2: What – if anything – about the quoted bit below would need to be changed to accurately describe the approach you described above?

"If we calculate the YoY change in the combined (iOS + Android) constructive activation rate by comparing the average 3-month rolling CA rate in June 2024 (April, May, June 2024) to the average 3-month rolling CA rate in June 2025 (April, May, June 2025), we notice a 23% YoY increase."

Note: Year-over-year data not available for iOS and Android breakdowns as this data is scrubbed after 90 days.
Mobile app rates continue to average around 2%; however, we have observed YoY increases. Constructive activation rates increased from 2.2% in June 2024 to 2.5% in June 2025.

Question #3: To be doubly sure, would it be accurate for me to understand the 2.2% → 2.5% y-o-y increase as a .03% absolute increase or 13.6% relative increase in the aggregate constructive activation rate of iOS and Android, combined?

Question #4: what is the "closest" we could get to reporting on a YoY change in the constructive activation rate on iOS specifically?

Question #1: What – if anything – about the quoted bit below would need to be changed to accurately describe the approach you described above?
"To smooth out the influence of specific events, this analysis calculates YoY changes in mobile web constructive activation rates by comparing the average 3-month rolling CA rate in June 2024 (April, May, June 2024) to the average 3-month rolling CA rate in June 2025 (April, May, June 2025). When comparing these 2024 and 2025 rolling averages, we observe no change in the constructive activation rate."

The above summary is accurate. No changes.
Note: If we did not use the 3-month rolling average, we observed a -5.7% Y0Y decrease in mobile web constructive activation rates directly comparing June 2024 through June 2024 but this is mostly because we observed an isolated spike in activation rates last June 2024 that was not sustained.

Question #2: What – if anything – about the quoted bit below would need to be changed to accurately describe the approach you described above?
"If we calculate the YoY change in the combined (iOS + Android) constructive activation rate by comparing the average 3-month rolling CA rate in June 2024 (April, May, June 2024) to the average 3-month rolling CA rate in June 2025 (April, May, June 2025), we notice a 23% YoY increase."

This is accurate as well.
The 3-month rolling CA rate in June 2024 was 2.08% and the 3-month rolling CA rate in June 2025 was 2.55%. Since the rates are so low this equates to a 23% YoY relative increase (0.47 percentage point absolute increase). When sharing these results, I'd recommend noting the calculated CA rates or absolute percentage point increase as well to give context to the 23% YoY increase.

Question #3: To be doubly sure, would it be accurate for me to understand the 2.2% → 2.5% y-o-y increase as a .03% absolute increase or 13.6% relative increase in the aggregate constructive activation rate of iOS and Android, combined?

Yes, if we don't use the 3-month rolling average, there was a 0.3 percentage point absolute YoY increase or 13.6% relative YoY increase in combined iOS and Android constructive activation rates.

Question #4: what is the "closest" we could get to reporting on a YoY change in the constructive activation rate on iOS specifically?

We unfortunately don't have iOS-specific data for June 2024 to calculate a direct YoY Change.

The last overall iOS-specific constructive activation rate I can find is January 2025, where it was 1.22%. We've seen a 47.5% relative increase (a 0.6 percentage point increase) comparing January 2025 to June 2025; however, this would include impacts from seasonal factors as well.

As another reference, @SNowick_WMF shared that the iOS Alt Text Experiment calculated a constructive activation baseline of 1.99% for August 2024. June 2025 had a slight decline compared to that rate; however, this also includes seasonal impacts and possible influences from specific events since we can't calculate the rolling average.

I"m resolving this task as requirements have been met and shared in T397876#10986094 but please reopen if needed.