I've completed the initial investigation into the effects of how load times affect Homepage engagements.

To keep this task constrained, I used a very simple approach to data gathering: I used the week prior to the bug going into effect (2024-06-11 through 2024-06-17) and compared that to the first week of the bug being present (2024-06-18 through 2024-06-24). I used data from all wikis except testwiki, and split the data up by platform (desktop and mobile web). Given this simple method, I have not attempted to determine if there's a *significant* change between the weeks, but instead try to interpret the findings based on the relative differences.

Summary of findings

1. Homepage visits: no substantial difference (-0.4%), but opposing trends between desktop and mobile web. Desktop decreases (-0.9%) whereas mobile web increases (+0.9%). Visits on desktop are roughly 2x that of mobile web, btw.

2. Homepage module impressions: meaningful decrease (-3.8%). The trends are similar on both platforms (desktop -4.4%, mobile web -2.5%), but the mobile web effect might be dampened by the slight increase in visits.

3. Module interactions: substantial decrease (-12.8%). The effect is larger on desktop (-14.2%), where most of our interactions happen, but still substantial on mobile web (-9.3%).

4. Clicks on Suggested Edits tasks: substantial decrease (-7.1%). Here the effect is larger on mobile web (-13.7%) than on desktop (-5.6%).

5. Completed Structured Task edits: overall no difference (-0.7%) but opposing trends between desktop and mobile. On mobile web we see a substantial decrease (-17.8%) while on desktop there's an increase (+4.5%).

Detailed findings

Homepage visits

We do not find a substantial difference in the number of Homepage visits. Overall, the pre/post difference is a decrease from 148.3k to 147.8k visits (-0.5k or -0.4%). We also do not find a substantial difference when splitting this up by platform, but we find opposing trends. On desktop there's a decrease from 106.7k to 105.8k visits (-0.9k or -0.9%), while on mobile web there's an increase from 42.6k to 43.0k (+0.4k or +0.9%).

Homepage module impressions

We find a meaningful decrease in the number of Homepage module impressions. Overall, we find a decrease pre/post from 604.5k to 581.4k (-23.1k or -3.8%). Split by platform, the decrease is larger on desktop than on mobile web. Desktop goes from 426.1k pre-bug to 407.6k post-bug (-18.6k or -4.4%), while mobile web goes from 178.3k to 173.9k (-4.5k or -2.5%).

Homepage module interactions

We find a substantial decrease in the number of interactions with Homepage modules. Overall the number of interactions decrease from 322.1k to 280.9k (-41.2k or -12.8%). Split by platform we see a larger relative decrease on desktop (227.6k to 195.3k, -32.4k or -14.2%) than on mobile web (94.4k to 85.6k, -8.8k or -9.3%).

Clicks on Suggested Edits tasks

We find a substantial decrease in the number of clicks on Suggested Edits tasks. Overall we find a reduction in clicks from 10.5k to 9.7k (-0.7k or -7.1%). Split by platform we find a larger relative difference on mobile web (1.9k to 1.7k, -0.3k or -13.7%) compared to desktop (8.6k to 8.1k, -0.5k or -5.6%).

Completed Structured Task edits

We find a substantial decrease in the number of completed Structured Task edits but on the mobile web platform. On desktop we see an increase, and overall the trend is flat (5.8k pre/post, -0.0k, -0.7%). On mobile web there's a decrease from 1.4k to 1.1k (-0.2k or -17.8%), on desktop there's an increase from 4.5k to 4.7k (+0.2k or +4.5%).

Thank you so much! I'm sorry for responding late, distracted with other matters.

While the general trend is in the general direction of what we would expect from Special:Homepage getting much slower.

However, I'm confused by the "5. Completed Structured Task edits" increasing on desktop by 4.5%.
While we probably don't have the data/resources to look into this more closely, would it still be possible to speculate what is going on there? Is this within the realm of the normal weekly fluctuation and so probably just noise?