| SNowick_WMF | |
| Aug 24 2026, 10:30 PM |
| F104372775: Screenshot 2026-09-22 at 4.06.27 PM.png | |
| Tue, Sep 22, 11:22 PM |
Analysis Date Range: 2026-08-28 - 2026-09-22
experiment.enrolled = 'ios-home-feed'
experiment.assigned IN ('treatment', 'control')
Prod builds:
6169-r-2026-09-04
6169-r-2026-09-04
6175-r-2026-09-15
KRS: See EPIC https://phabricator.wikimedia.org/T427526
Dates
Release into Beta: August 20, 2026
Release into Production: August 26, 2026
End of test: September 22, 2026
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T427526 [Epic] Scale Home Feed to iOS (Explore Feed Revamp) | |||
| Open | SNowick_WMF | T435885 iOS For You Feed AB Test Analysis |
We monitored this AB Test in Growthbook SN will conduct Measurement Plan analysis and report here at end of experiment 30 Day duration.
Summary: The "For You" feed revamp in iOS (ios-home-feed) was stopped as Won. It ran Aug 31–Sep 22, 2026, on about 3.16 million app install IDs. We have enough evidence that the redesigned feed slightly improves the goal metric (second week app user retention) and there's no evidence that it hurts any metric. User enrollment also tapered off and we're unlikely to get any larger sample sizes. At this point the decision is pretty clear to release the redesigned feed to everyone and just monitor our metrics post-release (@mpopov authored this conclusion)
Results:
| Metric | Treatment Chance to Win | CI |
|---|---|---|
| Second week app user retention | 91.1% | |
| Daily app open rate | 99% | 95% CI [0.037 %, 0.398%] Draw: this occurs when the % Change is smaller than the metric's min change (0.5%) |
| Second day app user retention | 52.8% | |
The iOS Home Feed treatment looks slightly positive, with no clear retention win and a daily-open lift too small to count:
Overall: no harm on retention, a tiny increase in daily opens
iOS For You Feed Analysis Results
Data
Key results
KR 1.1 Key Metric: 8% increase in logged-out second-week return for users in the variant group compared to control group
Result: We saw a -0.1% decrease in 2nd Week Retention Rate for Treatment compared to Control. 2nd week retention difference is not statistically significant, however we did not meet the goal metric.
Guardrails
GR 2.1 No more than 3% of users turn off all modules and leave them off over course of experiment
Result: Only .03% of Treatment users turned off (and did not re-enable) For You modules
GR 2.2 No more than 10% increase in non-retained new users within the Home feed group compared to Explore
Result: We did not see an increase in Treatment non-retained new users. Treatment churn rate was lower than control, 32.2% versus 34.1%, a 5.6% relative decrease
Curiosities
CR 3.1 What share of users updated their default feed to "For you"?
Result: 17% of users who saw the Set Feed Order screen opted to set For You first
CR 3.2 Do we see a difference in engagement from users who configured interests vs those that skipped?
Result: We saw a drastic increase (+1050%) in Avg Feed Engagement Rate for Interest Selected users compared to the Default/Skipped Select users
CR 3.3 Which modules in the Home feed drive the most engagement (as a share of each tab)?
"Result: The Based on Interests Module was the most frequently engaged module at 61% of all engagement events, the Continue Reading card had the least engagement of 11% of total.
When split by Interests Selected/Non Selected the most engaged module changes to BecauseYouRead for Non Interests Selected