Page MenuHomePhabricator

Thumbnails failing to render sporadically (ERR_CONNECTION_CLOSED or ERR_SSL_BAD_RECORD_MAC_ALERT)
Closed, ResolvedPublic

Description

On many pages (especially category pages) my browser fails to generae a bunch of thumbnails, forcing me to refresh/purge my tabs.

https://upload.wikimedia.org/wikipedia/en/thumb/3/37/Photodisc_website_2000.png/220px-Photodisc_website_2000.png Failed to load resource: net::ERR_CONNECTION_CLOSED

I was thinking that the servers might just be having some issues, but now after 2-3 days, I'm starting to think we got issues.

I'm on a Chromebook vith ChromeOS and Chrome as browser.

Details

Related Gerrit Patches:
operations/software/nginx : wmf-1.11.4remove readahead patch

Event Timeline

Restricted Application added subscribers: Matanya, Aklapper. · View Herald TranscriptOct 23 2016, 12:42 PM
Yann added a subscriber: Yann.EditedOct 23 2016, 4:25 PM

I have this issue on the English and French WP. Chrome on Win7.

Happends for me too on en.wikipedia.org

Urbanecm added a subscriber: Urbanecm.EditedOct 23 2016, 4:35 PM

I can't see any problem from my side. I use Google Chrome, Windows 10. There is no problem for me with Edge too.

I can't see any problem from my side. I use Google Chrome, Windows 10. There is no problem for me with Edge too.

I used Internet Explorer, haven't used edge or chrome yet.

It goes green in Microsoft edge.

In chrome, it doesn't display at all.

Paladox triaged this task as Unbreak Now! priority.Oct 23 2016, 4:52 PM

Changing to ubreak since users are reporting it in #wikimedia-tech, #wikipedia-en

I have also confirmed the problem myself.

I also asked if this task could be made unbreak.

<paladox> Can i set the task to unbreak please? Since other users are reporting it and it is a production wiki?
<Urbanecm> I think you can...

Restricted Application added subscribers: Jay8g, Luke081515, TerraCodes. · View Herald TranscriptOct 23 2016, 4:52 PM
Aklapper lowered the priority of this task from Unbreak Now! to High.Oct 23 2016, 5:07 PM

Can someone please provide the error message (and the browser, browser version, operating system) shown and the URL/web address when opening the thumbnail image itself (not the page where it is embedded) in the browser? Thanks!

Cannot reproduce here. :(

Urbanecm moved this task from Backlog to Watching on the User-Urbanecm board.
Josve05a added a comment.EditedOct 23 2016, 6:38 PM

Can someone please provide the error message (and the browser, browser version, operating system) shown and the URL/web address when opening the thumbnail image itself (not the page where it is embedded) in the browser? Thanks!
Cannot reproduce here. :(

Console logs:

Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/7/74/%22Be_alert_for_t…f/lossy-page1-87px-%22Be_alert_for_turning_cars%22_-_NARA_-_513925.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/9/9f/%22Be_careful_wha…at_you_say_or_write._Please_get_there_and_back.%22_-_NARA_-_513926.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/5/5e/%22Be_doubly_aler…13927.jpg/93px-%22Be_doubly_alert_crossing_at_night%22_-_NARA_-_513927.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/b/be/%22Be_doubly_aler…page1-93px-%22Be_doubly_alert_crossing_at_night%22_-_NARA_-_513927.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/c/ce/%22Be_kind_to_you…-_NARA_-_513928.jpg/86px-%22Be_kind_to_your_stomach%22_-_NARA_-_513928.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/b/b3/%22Be_kind_to_you…tif/lossy-page1-86px-%22Be_kind_to_your_stomach%22_-_NARA_-_513928.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/0/02/%22Be_Patriotic_s…otic_sign_your_country%27s_pledge_to_save_the_food.%22_-_NARA_-_512548.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/2/2c/%22Be_Patriotic_s…_sign_your_country%27s_pledge_to_save_the_food.%22_-_NARA_-_512548.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/5/51/%22Be_Patriotic_s…27s_pledge_to_save_the_food.%22%2C_ca._1917_-_ca._1919_-_NARA_-_512497.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/9/9d/%22Be_proud_your_…_NARA_-_513929.jpg/84px-%22Be_proud_your_delivering%22_-_NARA_-_513929.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/6/6e/%22Be_proud_your_…if/lossy-page1-84px-%22Be_proud_your_delivering%22_-_NARA_-_513929.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/6/6c/%22Be_ready_for_y…your_war_service-Join_the_Red_Cross_Student_Reserve%22_-_NARA_-_513930.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/5/5b/%22Be_ready_for_y…_war_service-Join_the_Red_Cross_Student_Reserve%22_-_NARA_-_513930.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/7/73/%22BE_READY_TO_JU…8.jpg/120px-%22BE_READY_TO_JUMP_INTO_THEIR_SHOES%5E%22_-_NARA_-_535598.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/7/77/%22BE_READY_TO_JU…1-120px-%22BE_READY_TO_JUMP_INTO_THEIR_SHOES%5E%22_-_NARA_-_535598.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/a/ad/%22Be_silent%22_-_NARA_-_513931.jpg/92px-%22Be_silent%22_-_NARA_-_513931.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/a/a1/%22Be_silent%22_-_NARA_-_513931.tif/lossy-page1-92px-%22Be_silent%22_-_NARA_-_513931.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/0/0a/%22Be_Smart_Act_Dumb%22_-_NARA_-_514912.jpg/89px-%22Be_Smart_Act_Dumb%22_-_NARA_-_514912.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/7/74/%22Be_Smart_Act_D…14912.tif/lossy-page1-89px-%22Be_Smart_Act_Dumb%22_-_NARA_-_514912.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/7/71/%22Be_smart..Act_…b%5E_-_NARA_-_513932.jpg/86px-%22Be_smart..Act_dumb%5E_-_NARA_-_513932.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/f/f9/%22Be_smart..Act_…3932.tif/lossy-page1-86px-%22Be_smart..Act_dumb%5E_-_NARA_-_513932.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/f/fc/%22Be_sure_it%27s…x-%22Be_sure_it%27s_right%2C_check_and_double_check%22_-_NARA_-_513934.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/d/d6/%22Be_sure_it%27s…2Be_sure_it%27s_right%2C_check_and_double_check%22_-_NARA_-_513934.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/1/12/%22Be_sure_the_dr…RA_-_513933.jpg/87px-%22Be_sure_the_driver_sees_you%22_-_NARA_-_513933.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/e/eb/%22Be_sure_the_dr…lossy-page1-87px-%22Be_sure_the_driver_sees_you%22_-_NARA_-_513933.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/8/8c/%22Be_with_him_at…_513937.jpg/88px-%22Be_with_him_at_every_mail_call.%22_-_NARA_-_513937.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/0/06/%22Be_with_him_at…y-page1-88px-%22Be_with_him_at_every_mail_call.%22_-_NARA_-_513937.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/c/ce/%22Bear_Poster%22…939.jpg/120px-%22Bear_Poster%22_%28can%27t_you_read%29_-_NARA_-_513939.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/8/8b/%22Bear_Poster%22…ge1-120px-%22Bear_Poster%22_%28can%27t_you_read%29_-_NARA_-_513939.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/6/66/%22Bearing_a_burden%22_-_NARA_-_513940.jpg/86px-%22Bearing_a_burden%22_-_NARA_-_513940.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/0/0c/%22Bearing_a_burd…513940.tif/lossy-page1-86px-%22Bearing_a_burden%22_-_NARA_-_513940.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/8/8e/%22Beat_back_the_…8.jpg/80px-%22Beat_back_the_HUN_with_LIBERTY_BONDS.%22_-_NARA_-_512638.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/e/e4/%22Beat_back_the_…1-80px-%22Beat_back_the_HUN_with_LIBERTY_BONDS.%22_-_NARA_-_512638.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/3/35/%22Beat_the_boll_…u-_not_the_weevils-_get_the_crop._Get_a_good_cotton..._-_NARA_-_512572.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/5/5f/%22Beat_the_boll_…ot_the_weevils-_get_the_crop._Get_a_good_cotton..._-_NARA_-_512572.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/e/e0/%22Beat_the_promise%22_-_NARA_-_513941.jpg/120px-%22Beat_the_promise%22_-_NARA_-_513941.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/9/91/%22Beat_the_promi…13941.tif/lossy-page1-120px-%22Beat_the_promise%22_-_NARA_-_513941.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/0/07/%22Beat_the_skin_…e%22_-_NARA_-_513942.jpg/86px-%22Beat_the_skin_game%22_-_NARA_-_513942.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/d/df/%22Beat_the_skin_…3942.tif/lossy-page1-86px-%22Beat_the_skin_game%22_-_NARA_-_513942.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/5/52/%22Beauty_Operato…lling_machine%2C_US_Navy_Yard%2C_Mare_Island%2C_CA.%22_-_NARA_-_296890.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/8/8a/%22Beauty_Operato…g_machine%2C_US_Navy_Yard%2C_Mare_Island%2C_CA.%22_-_NARA_-_296890.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/b/ba/%22Because_somebo…-_NARA_-_513669.jpg/86px-%22Because_somebody_talked%22_-_NARA_-_513669.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/f/fd/%22Because_somebo…tif/lossy-page1-86px-%22Because_somebody_talked%22_-_NARA_-_513669.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/0/05/%22Become_a_marin…13586.jpg/82px-%22Become_a_marine_officer_candidate%22_-_NARA_-_513586.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/d/dc/%22Become_a_marin…page1-82px-%22Become_a_marine_officer_candidate%22_-_NARA_-_513586.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/1/1e/%22Become_a_nurse…jpg/95px-%22Become_a_nurse_-_Your_country_needs_you%22_-_NARA_-_513583.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/d/d6/%22Become_a_nurse…95px-%22Become_a_nurse_-_Your_country_needs_you%22_-_NARA_-_513583.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/d/d9/%22Before_crossin…13943.jpg/92px-%22Before_crossing%2C_look_both_ways%22_-_NARA_-_513943.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/9/93/%22Before_crossin…page1-92px-%22Before_crossing%2C_look_both_ways%22_-_NARA_-_513943.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/8/86/%22Before_the_War%22_-_NH_78627-KN.jpg/95px-%22Before_the_War%22_-_NH_78627-KN.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/f/fe/%22Before.%22_Nev…44.jpg/84px-%22Before.%22_Never_give_a_germ_a_break%5E_-_NARA_-_513944.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/7/7c/%22Before.%22_Nev…e1-84px-%22Before.%22_Never_give_a_germ_a_break%5E_-_NARA_-_513944.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/2/2b/%22Beggar_Boy%22_-_NARA_-_558953.jpg/90px-%22Beggar_Boy%22_-_NARA_-_558953.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/0/0a/%22Beggar_Boy%22_…ARA_-_558953.tif/lossy-page1-90px-%22Beggar_Boy%22_-_NARA_-_558953.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/d/da/%22Beggars%22%2C_1963_-_NARA_-_558937.jpg/89px-%22Beggars%22%2C_1963_-_NARA_-_558937.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/a/a2/%22Beggars%22%2C_…_558937.tif/lossy-page1-89px-%22Beggars%22%2C_1963_-_NARA_-_558937.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/d/d0/%22Behind_the_eag…3948.jpg/95px-%22Behind_the_eagel_stand_the_forests%22_-_NARA_-_513948.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/f/f2/%22Behold_Joe_Dop…ride%27%5E_The_sequel_of_this_is_some_duds_and_near..._-_NARA_-_514700.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/c/c8/%22Behold_Joe_Dop…%27%5E_The_sequel_of_this_is_some_duds_and_near..._-_NARA_-_514700.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/c/cf/%22Belgium%27s_To…x-%22Belgium%27s_Tomorrow_Must_Depend_On_Our_Today.%22_-_NARA_-_512565.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/1/1a/%22Belgium%27s_To…2Belgium%27s_Tomorrow_Must_Depend_On_Our_Today.%22_-_NARA_-_512565.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/4/44/%22BELLE_OF_LOUIS…EEL_STEAMBOAT_OWNED_BY_LOUISVILLE_AND_JEFFERSON_COUNTY_-_NARA_-_543970.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/8/87/%22BELLE_OF_LOUIS…STEAMBOAT_OWNED_BY_LOUISVILLE_AND_JEFFERSON_COUNTY_-_NARA_-_543970.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/c/c5/%22Benga%22_-_NARA_-_559038.jpg/96px-%22Benga%22_-_NARA_-_559038.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/f/fa/%22Benga%22_-_NARA_-_559038.tif/lossy-page1-96px-%22Benga%22_-_NARA_-_559038.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/4/4f/%22Benin_Wrestlers%22_-_NARA_-_558984.jpg/96px-%22Benin_Wrestlers%22_-_NARA_-_558984.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/3/34/%22Benin_Wrestler…_558984.tif/lossy-page1-96px-%22Benin_Wrestlers%22_-_NARA_-_558984.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/b/b3/%22Benny_Bungles_…14px-%22Benny_Bungles_Again%5E_America%27s_Alertmen%22_-_NARA_-_513951.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/c/cf/%22Benny_Bungles_…-%22Benny_Bungles_Again%5E_America%27s_Alertmen%22_-_NARA_-_513951.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/4/49/%22Bertha_Stallwo…tridge_case_at_Frankford_Arsenal.%22%2C_ca._1941_-_ca._-_NARA_-_535805.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/9/94/%22Bertha_Stallwo…ge_case_at_Frankford_Arsenal.%22%2C_ca._1941_-_ca._-_NARA_-_535805.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/d/d5/%22Best_and_Safes…or_everybody_is_Liberty_bonds_paying_4%5E_interest.%22_-_NARA_-_512673.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/a/a3/%22Best_and_Safes…verybody_is_Liberty_bonds_paying_4%5E_interest.%22_-_NARA_-_512673.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/f/f0/%22Better_a_live_…59.jpg/77px-%22Better_a_live_cluck_than_a_dead_duck%22_-_NARA_-_513959.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/c/c3/%22Better_a_live_…e1-77px-%22Better_a_live_cluck_than_a_dead_duck%22_-_NARA_-_513959.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/e/e8/%22Better_health_…NARA_-_513960.jpg/87px-%22Better_health_through_FSA%22_-_NARA_-_513960.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/5/5d/%22Better_health_…f/lossy-page1-87px-%22Better_health_through_FSA%22_-_NARA_-_513960.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/9/9c/%22Beula_Ogle_pre…Gatlinburg%2C_Tennessee._She_is_a_new_weaver_at_the..._-_NARA_-_532765.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/5/5a/%22Beula_Ogle_pre…inburg%2C_Tennessee._She_is_a_new_weaver_at_the..._-_NARA_-_532765.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/f/f0/%22Beulah_Wilhelm…kill_and_experience_are_required_for_this_operation..._-_NARA_-_532751.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/8/89/%22Beulah_Wilhelm…_and_experience_are_required_for_this_operation..._-_NARA_-_532751.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/1/1a/%22Beware_of_carb…NARA_-_513966.jpg/86px-%22Beware_of_carbon_monoxide%22_-_NARA_-_513966.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
https://upload.wikimedia.org/wikipedia/commons/thumb/0/05/%22Beware_of_carb…f/lossy-page1-86px-%22Beware_of_carbon_monoxide%22_-_NARA_-_513966.tif.jpg Failed to load resource: net::ERR_SSL_BAD_RECORD_MAC_ALERT
index.php:1666 Gadget "CollapsibleTemplates" styles loaded twice. Migrate to type=general. See <https://phabricator.wikimedia.org/T42284>.
index.php:1666 Gadget "Long-Image-Names-in-Categories" styles loaded twice. Migrate to type=general. See <https://phabricator.wikimedia.org/T42284>.
index.php:1666 Gadget "WatchlistNotice" styles loaded twice. Migrate to type=general. See <https://phabricator.wikimedia.org/T42284>.
index.php:1666 Gadget "AjaxQuickDelete" styles loaded twice. Migrate to type=general. See <https://phabricator.wikimedia.org/T42284>.
index.php:1666 Gadget "Cat-a-lot" styles loaded twice. Migrate to type=general. See <https://phabricator.wikimedia.org/T42284>.
content_script.js:64 init 2
load.php?debug=false&lang=en&modules=jquery.ui.autocomplete%2Cbutton%2Ccore%2Cdialog%2Cdraggable%2C…:77 This page is using the deprecated ResourceLoader module "jquery.ui.position".
load.php?debug=false&lang=en&modules=jquery.ui.autocomplete%2Cbutton%2Ccore%2Cdialog%2Cdraggable%2C…:121 This page is using the deprecated ResourceLoader module "jquery.ui.widget".
load.php?debug=false&lang=en&modules=jquery.ui.autocomplete%2Cbutton%2Ccore%2Cdialog%2Cdraggable%2C…:20 This page is using the deprecated ResourceLoader module "jquery.ui.core".
Please use "mediawiki.ui.button" or "oojs-ui" instead.
load.php?debug=false&lang=en&modules=ext.gadget.libAPI|jquery.tipsy&skin=vector&version=1fazz7v:25 This page is using the deprecated ResourceLoader module "jquery.tipsy".
Josve05a renamed this task from Thumbnails failing to render sporadically (ERR_CONNECTION_CLOSED) to Thumbnails failing to render sporadically (ERR_CONNECTION_CLOSED or ERR_SSL_BAD_RECORD_MAC_ALERT).Oct 23 2016, 6:39 PM

@Paladox: Please see T148917#2736825 for information that would be helpful here (what's shown when opening the thumbnail images themselves, not pages displaying them).

Oh, the images show when I go to upload.wikimedia.org. But on the wiki them selfs it dosen't

This comment was removed by Urbanecm.
BBlack added a subscriber: BBlack.Oct 23 2016, 6:48 PM

So, one of the possible factors here is various HTTPS-interfering proxies or filters. Often this is from some kind of anti-virus or anti-malware tool, either on the client machine or on the local wifi router/gateway. There are similar reports for various sorts of this software, and the issues can also be UA-specific (e.g. "only affects Safari 10 + Cisco AMP", or "only affected IE11 + Avast WebShield", etc). Similarly, anything in a corporate office on office-controlled clients might have similar issues from corporate TLS proxies.

Can we get some confirmation whether the affected UAs are cleanly-connected? Are AV/malware scanners (on the client and/or router) fully updated? Can they be disabled to test their interference? etc.

Josve05a added a comment.EditedOct 23 2016, 6:50 PM

So, one of the possible factors here is various HTTPS-interfering proxies or filters. Often this is from some kind of anti-virus or anti-malware tool, either on the client machine or on the local wifi router/gateway. There are similar reports for various sorts of this software, and the issues can also be UA-specific (e.g. "only affects Safari 10 + Cisco AMP", or "only affected IE11 + Avast WebShield", etc). Similarly, anything in a corporate office on office-controlled clients might have similar issues from corporate TLS proxies.
Can we get some confirmation whether the affected UAs are cleanly-connected? Are AV/malware scanners (on the client and/or router) fully updated? Can they be disabled to test their interference? etc.

I am on a Chromebook, which does not support anti-virus softwares. (But it might be office-controlled somehow) Just saying. Might be AdBlocker which may interfere...

Paladox added a comment.EditedOct 23 2016, 6:52 PM

So, one of the possible factors here is various HTTPS-interfering proxies or filters. Often this is from some kind of anti-virus or anti-malware tool, either on the client machine or on the local wifi router/gateway. There are similar reports for various sorts of this software, and the issues can also be UA-specific (e.g. "only affects Safari 10 + Cisco AMP", or "only affected IE11 + Avast WebShield", etc). Similarly, anything in a corporate office on office-controlled clients might have similar issues from corporate TLS proxies.
Can we get some confirmation whether the affected UAs are cleanly-connected? Are AV/malware scanners (on the client and/or router) fully updated? Can they be disabled to test their interference? etc.

I had to uninstall Norton security recently due to a bug in the latest windows 10 insider preview builds which broke third party anti virus.

The only one that I'm using is Microsoft Defender since that is turned on by default.

But this has never happened before.

Also the router I'm using is a bt home hub (isp provided), it doesn't allow you to touch the software but allows you to configure the ports.

But the bt home hub is no longer being updated since bt smart hub is released now.

I changed WiFi to my phones personal hotspot, and that didn't chnage anything.

I've reproduced this now, at least once. I took a lot of retries. I used one of the category links from earlier, specifically https://commons.wikimedia.org/w/index.php?title=Category:Media_with_locations&filefrom=%22+Dogsbody%22+%2C+The+Broadleys+%2C+Claycross+%286331958120%29.jpg#mw-category-media . Eventually, a number of the images failed to load, with the ssl bad record mac errors in the console. This was with Chromium 53 on Linux, and with my /etc/hosts pointing traffic towards esams even though I'm in the US.

I doubt that it's actually esams-specific, but it's possible.

T148830 describes a related caching issue
T145811 and T93041 describe a missing thumbnails issue, although the latter was fixed

ema added a subscriber: ema.Oct 24 2016, 10:32 AM
ema moved this task from Triage to General on the Traffic board.Oct 24 2016, 11:26 AM
ema added a comment.Oct 24 2016, 11:31 AM

As @BBlack mentioned on IRC, the reports tend to happen in esams at peak request-load and upload has more data to transfer per request leading to higher parallelism, so this issue might be related to TCP port issues described in T107749.

Can anyone still repro this issue? We've done a bunch of debugging today, and deployed a possible workaround.

Nope carn't reproduce it yet.

BBlack moved this task from General to Caching on the Traffic board.Oct 24 2016, 11:48 PM
BBlack moved this task from Caching to TLS on the Traffic board.

@Paladox: Please see T148917#2736825 for information that would be helpful here (what's shown when opening the thumbnail images themselves, not pages displaying them).

To answer that: URL seemed normal, error message was (on Firefox) "Content-Encoding-Fehler: […] eine ungültige oder unbekannte Form der Kompression verwendet."

The following was tested on for the first report on dewiki, and not retested on the later reports (so certain it still applies to this issue): HTTP Headers were identical to a working image, even the content size header seemed normal (several KiB), but it didn't actually transfer. Strangely curl did work though.

I don't know any image for which this is still happening right now, but I haven't seen this anywhere aside from the occurences mentioned at dewiki:WP:FzW either.

Content-Encoding issues are a separate thing unrelated to this ticket. They're over here: T148830

Gilles added a subscriber: Gilles.Oct 25 2016, 4:16 PM
greg added a subscriber: greg.Oct 25 2016, 4:17 PM

Mentioned in SAL (#wikimedia-operations) [2016-10-31T12:08:07Z] <bblack> upgrading nginx to 1.11.4-1+wmf13 on cache_upload - T148917

Mentioned in SAL (#wikimedia-operations) [2016-10-31T12:32:14Z] <bblack> upgrading nginx to 1.11.4-1+wmf13 on cache_misc - T148917

Can anyone still repro this issue? We've done a bunch of debugging today, and deployed a possible workaround.

The above workaround back on the 24th was to downgrade to our previous software revs for TLS termination. As of today (SAL entry above), the upload servers are now re-upgraded to the newer software stack again, with a more-precise workaround for the specific bug in this ticket.

If you were able to reproduce this issue before, can you please re-attempt now and report back?

Thanks!

Mentioned in SAL (#wikimedia-operations) [2016-11-03T13:42:16Z] <bblack> cp*: upgrade libssl1.1 to 1.1.0b-1+wmf2 (but no nginx restart yet) - T144626 - T148917

Mentioned in SAL (#wikimedia-operations) [2016-11-03T13:49:25Z] <bblack> cache_maps + cache_misc: nginx lossless restarts for libssl update - T144626 - T148917

Mentioned in SAL (#wikimedia-operations) [2016-11-03T13:56:14Z] <bblack> cache_upload: nginx lossless restarts for libssl update - T144626 - T148917

Mentioned in SAL (#wikimedia-operations) [2016-11-03T14:08:14Z] <bblack> cache_text: nginx lossless restarts for libssl update - T144626 - T148917

Change 320114 had a related patch set uploaded (by BBlack):
remove readahead patch

https://gerrit.wikimedia.org/r/320114

Change 320114 merged by BBlack:
remove readahead patch

https://gerrit.wikimedia.org/r/320114

Mentioned in SAL (#wikimedia-operations) [2016-11-10T00:46:16Z] <bblack> nginx-1.11.4-1+wmf14 uploaded to carbon jessie-wikimedia (only deployed to cp1008 for now) - T93927 - T148917 - T144523

BBlack closed this task as Resolved.Nov 15 2016, 2:05 PM
BBlack claimed this task.

This was worked-around a few different ways over time, and now has a final (upstreamed) fix that we're actually using instead of papering over, so we should be done here.

Just got another Failed to load resource: net::ERR_CONNECTION_CLOSED on https://commons.wikimedia.org/wiki/Category:Tony_Blinken
Might have just been a one-off, or a regression...

Can't be a regression of the specific TLS bug we had here.