| Shawn | |
| Oct 30 2020, 12:22 PM |
| F32419656: Screenshot_20201030_145412.png | |
| Oct 30 2020, 6:55 PM |
| F32419388: Screenshot from 2020-10-30 14-02-15.png | |
| Oct 30 2020, 1:04 PM |
Hello, there is a very long response time on frwiki main page (https://fr.wikipedia.org/wiki/Wikip%C3%A9dia:Accueil_principal).
When I'm logged on, it takes around 20 seconds to receive the first byte from the server.
This issue doesn't appear when I'm logged out.
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Reduce size of frwiki featuredfeeds to default | operations/mediawiki-config | master | +0 -1 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | CDanis | T266865 Very long response time on frwiki main page | |||
| Resolved | Legoktm | T266900 FeaturedFeeds loads all feed content just to output the feed URLs on the main page |
Same issue here.
Discussion on frwiki village pump: https://fr.wikipedia.org/wiki/Wikip%C3%A9dia:Le_Bistro/30_octobre_2020#Temps_de_chargement_de_la_page_d'accueil
(In theory I'd point to https://wikitech.wikimedia.org/wiki/Reporting_a_connectivity_issue but I don't see which of those items might be helpful here.)
This isn't limited to just esams; it is in fact happening across all cache clusters.
All of my requests took at least 19 seconds. Some of them took ~38 seconds.
Additionally you don't need to be logged in to reproduce this; other ways that produce a cache miss also trigger this behavior.
I will hazard a guess: something was changed recently that causes rendering the frwiki front page to trigger an RPC guarded by Poolcounter. Said RPC times out.
Change 637752 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/mediawiki-config@master] Reduce size of frwiki featuredfeeds to default
Change 637752 merged by jenkins-bot:
[operations/mediawiki-config@master] Reduce size of frwiki featuredfeeds to default
Mentioned in SAL (#wikimedia-operations) [2020-10-30T18:47:58Z] <cdanis> ✔️ cdanis@deploy1001.eqiad.wmnet /srv/mediawiki-staging 🕝☕ scap sync-file wmf-config/InitialiseSettings.php 'lower frwiki featured feeds limit 1a41ef634 T266865'
Mentioned in SAL (#wikimedia-operations) [2020-10-30T18:48:06Z] <cdanis@deploy1001> Synchronized wmf-config/InitialiseSettings.php: lower frwiki featured feeds limit 1a41ef634 T266865 (duration: 05m 14s)
Approx 23:00 on 28 Oct, the size of the featured feed for frwiki started to become too large to be stored as a value in our memcached.
Memcached error for key "WANCache:frwiki:featured-feeds:1:fr|#|v" on server "127.0.0.1:11213": ITEM TOO BIG
This means the feed contents were being recomputed on each main page render, even though (as pointed out by @Legoktm) the main page doesn't actually need the contents, only some metadata. https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/FeaturedFeeds/+/refs/heads/master/includes/FeaturedFeeds.php#116
(Followup AI: fix this :)
Long ago, frwiki's default feed length (in days) was set to 60, well above the default of 10.
We reverted the limit to the default.
Special thanks to @Legoktm and @Urbanecm for helping track this down!
I'm not sure what the historical reasoning behind this was, but as long as people open their feed readers at least once every 10 days, they shouldn't miss any content, so I would say the practical impact of this change will be pretty low.
Yes, the impact should only be "users reading the aggregated feed on fr.planet.wikimedia.org get fewer items included in that feed".
The limit of 60 _items per daily feed_ was reduced to the default of 10.
Left comments about it at https://fr.wikipedia.org/wiki/Wikip%C3%A9dia:Le_Bistro#Temps_de_chargement_de_la_page_d'accueil
