Page MenuHomePhabricator

Unexplained increase in thumbnail 500s
Closed, DeclinedPublic

Description

This the 500 rate specifically for varnish->swift requests for thumbnails (pattern doesn't exist in requests for originals):

2016-10-07-143726_1389x270_scrot.png (270×1 px, 46 KB)

The first non-zero sample in that graph is at 2016-08-16 09:00
The next notable bump up is somewhere in the early half of 2016-09-13
The rise to the current (highest) levels is somewhere in the early half of 2016-09-23

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Could be related to thumbor deployment, partial deployment was enabled on Sept 7th with https://gerrit.wikimedia.org/r/#/c/308746/ on Sept 13th on small wikis https://gerrit.wikimedia.org/r/#/c/310268/ and later in the day for commons with https://gerrit.wikimedia.org/r/#/c/310282/

Another data point, scrolling now the list of 500s I see a good chunk with size '0px' coming from user_agent": "Wikipedia/942 CFNetwork/808.0.2 Darwin/16.0.0" that could contribute to the increase

Wikipedia/942 CFNetwork/808.0.2 Darwin/16.0.0 is a user agent passed by the iOS app's WebView (which displays the article HTML). In some cases, the iOS app alters the requested thumbnail size, so it's possible there's a bug that's causing a 0px thumbnail to be requested.

I made a change to prevent the iOS app from requesting zero width thumbnails. Should go out with version 5.3.0

@JoeWalsh - sounds great! can we get a link to the change?

@JoeWalsh is there a timeline for 5.3.0 ? We're still seeing significant traffic for 0px requests

According to SoS, 5.3.0 iOS app has been shipped last week, so we should start seeing traffic for 0px requests dropping

Marostegui subscribed.

There is no way we can debug this anymore after 4 years :)

Aklapper changed the task status from Resolved to Declined.May 13 2020, 2:43 PM

Translates to declined to me as nothing was actively resolved :)