Page MenuHomePhabricator

Cached page previews not shown when refreshed
Open, HighPublic

Description

  1. Login to English Wikipedia and enable page previews.
  2. Open Chromium Developer Tools and click the network tab.
  3. Visit the Sonic the Hedgehog article.
  4. Hover over the "Sonic Spinball" (or a preview not requested previously) link. This is the initial request / 200 response:
Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
Request Method:GET
Status Code:200 
Remote Address:208.80.154.224:443
Referrer Policy:origin-when-cross-origin

Request:
:authority:en.wikipedia.org
:method:GET
:path:/api/rest_v1/page/summary/Sonic_Spinball
:scheme:https
accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9
cookie:CP=H2; GeoIP=US:TN:Cordova:35.14:-89.77:v4; loginnotify_prevlogins=2017-4bna7w-5u9tr14qqj8vtxax4uf44099fgftf9g; VEE=wikitext; enwikiUserID=13204772; enwikiUserName=Niedzielski; centralauth_User=Niedzielski; centralauth_Token=1f3ae7c802862ac7849ccb8288230417; forceHTTPS=true; optin=beta; WMF-Last-Access=09-Jan-2018; WMF-Last-Access-Global=09-Jan-2018; enwikiSession=97juvgbur2g4fep52c9q0tl4dd038vks; centralauth_Session=d5d697afafddf1edce736a8e083d64aa
referer:https://en.wikipedia.org/wiki/Sonic_the_Hedgehog
user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36
x-requested-with:XMLHttpRequest

Response:
accept-ranges:bytes
access-control-allow-headers:accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding
access-control-allow-methods:GET,HEAD
access-control-allow-origin:*
access-control-expose-headers:etag
age:0
cache-control:s-maxage=1209600, max-age=300
content-encoding:gzip
content-length:778
content-location:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
content-security-policy:default-src 'none'; frame-ancestors 'none'
content-type:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
date:Tue, 09 Jan 2018 16:21:57 GMT
etag:"815568267/8dd72aa6-e1fd-11e7-a66d-f2eb3c4dd428"
referrer-policy:origin-when-cross-origin
server:restbase1007
status:200
strict-transport-security:max-age=106384710; includeSubDomains; preload
vary:Accept-Encoding
via:1.1 varnish-v4, 1.1 varnish-v4
x-analytics:WMF-Last-Access=09-Jan-2018;WMF-Last-Access-Global=09-Jan-2018;https=1
x-cache:cp1066 miss, cp1066 miss
x-cache-status:miss
x-client-ip:73.252.38.252
x-content-security-policy:default-src 'none'; frame-ancestors 'none'
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-request-id:36848c22-f559-11e7-b6bd-f764a5a3c585
x-varnish:338894840, 181105768
x-webkit-csp:default-src 'none'; frame-ancestors 'none'
x-xss-protection:1; mode=block
  1. Move the cursor away to dismiss the preview.
  2. Go offline and hover over the same link. The response is now served from cache:
Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
Request Method:GET
Status Code:200  (from disk cache)
Remote Address:208.80.154.224:443
Referrer Policy:origin-when-cross-origin

Request:
Provisional headers are shown
Accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
Referer:https://en.wikipedia.org/wiki/Sonic_the_Hedgehog
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36
X-Requested-With:XMLHttpRequest

Response:
accept-ranges:bytes
access-control-allow-headers:accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding
access-control-allow-methods:GET,HEAD
access-control-allow-origin:*
access-control-expose-headers:etag
age:0
cache-control:s-maxage=1209600, max-age=300
content-encoding:gzip
content-length:778
content-location:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
content-security-policy:default-src 'none'; frame-ancestors 'none'
content-type:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
date:Tue, 09 Jan 2018 16:21:57 GMT
etag:"815568267/8dd72aa6-e1fd-11e7-a66d-f2eb3c4dd428"
referrer-policy:origin-when-cross-origin
server:restbase1007
status:200
vary:Accept-Encoding
via:1.1 varnish-v4, 1.1 varnish-v4
x-analytics:WMF-Last-Access=09-Jan-2018;WMF-Last-Access-Global=09-Jan-2018;https=1
x-cache:cp1066 miss, cp1066 miss
x-cache-status:miss
x-client-ip:73.252.38.252
x-content-security-policy:default-src 'none'; frame-ancestors 'none'
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-request-id:36848c22-f559-11e7-b6bd-f764a5a3c585
x-varnish:338894840, 181105768
x-webkit-csp:default-src 'none'; frame-ancestors 'none'
x-xss-protection:1; mode=block
  1. Move the cursor away to dismiss the preview.
  2. Wait at least five minutes.
  3. Go online and hover over the same link. The response is refreshed from the network as a 200:
Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
Request Method:GET
Status Code:200 
Remote Address:208.80.154.224:443
Referrer Policy:origin-when-cross-origin

Request:
:authority:en.wikipedia.org
:method:GET
:path:/api/rest_v1/page/summary/Sonic_Spinball
:scheme:https
accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9
cookie:CP=H2; GeoIP=US:TN:Cordova:35.14:-89.77:v4; loginnotify_prevlogins=2017-4bna7w-5u9tr14qqj8vtxax4uf44099fgftf9g; VEE=wikitext; enwikiUserID=13204772; enwikiUserName=Niedzielski; centralauth_User=Niedzielski; centralauth_Token=1f3ae7c802862ac7849ccb8288230417; forceHTTPS=true; optin=beta; WMF-Last-Access=09-Jan-2018; WMF-Last-Access-Global=09-Jan-2018; enwikiSession=97juvgbur2g4fep52c9q0tl4dd038vks; centralauth_Session=d5d697afafddf1edce736a8e083d64aa
if-none-match:"815568267/8dd72aa6-e1fd-11e7-a66d-f2eb3c4dd428"
referer:https://en.wikipedia.org/wiki/Sonic_the_Hedgehog
user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36
x-requested-with:XMLHttpRequest

Response:
accept-ranges:bytes
access-control-allow-headers:accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding
access-control-allow-methods:GET,HEAD
access-control-allow-origin:*
access-control-expose-headers:etag
age:470
cache-control:s-maxage=1209600, max-age=300
content-encoding:gzip
content-length:778
content-location:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
content-security-policy:default-src 'none'; frame-ancestors 'none'
content-type:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
date:Tue, 09 Jan 2018 16:29:47 GMT
etag:"815568267/8dd72aa6-e1fd-11e7-a66d-f2eb3c4dd428"
referrer-policy:origin-when-cross-origin
server:restbase1007
status:200
strict-transport-security:max-age=106384710; includeSubDomains; preload
vary:Accept-Encoding
via:1.1 varnish-v4, 1.1 varnish-v4
x-analytics:WMF-Last-Access=09-Jan-2018;WMF-Last-Access-Global=09-Jan-2018;https=1
x-cache:cp1066 hit/3, cp1066 miss
x-cache-status:hit-local
x-client-ip:73.252.38.252
x-content-security-policy:default-src 'none'; frame-ancestors 'none'
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-request-id:36848c22-f559-11e7-b6bd-f764a5a3c585
x-varnish:344684204 338894841, 198281743
x-webkit-csp:default-src 'none'; frame-ancestors 'none'
x-xss-protection:1; mode=block
  1. Go offline and hover over the same link. The request now fails:
Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
Referrer Policy:origin-when-cross-origin

Request:
Provisional headers are shown
Accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
Referer:https://en.wikipedia.org/wiki/Sonic_the_Hedgehog
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36
X-Requested-With:XMLHttpRequest
  1. Go online. If you issue more requests at this point, the responses will all be 200s.
  2. Refresh the page.
  3. Hover over the same link. The response is now a 304:
Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
Request Method:GET
Status Code:304 
Remote Address:208.80.154.224:443
Referrer Policy:origin-when-cross-origin

Request:
:authority:en.wikipedia.org
:method:GET
:path:/api/rest_v1/page/summary/Sonic_Spinball
:scheme:https
accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9
cookie:CP=H2; GeoIP=US:TN:Cordova:35.14:-89.77:v4; loginnotify_prevlogins=2017-4bna7w-5u9tr14qqj8vtxax4uf44099fgftf9g; VEE=wikitext; enwikiUserID=13204772; enwikiUserName=Niedzielski; centralauth_User=Niedzielski; centralauth_Token=1f3ae7c802862ac7849ccb8288230417; forceHTTPS=true; optin=beta; WMF-Last-Access=09-Jan-2018; WMF-Last-Access-Global=09-Jan-2018; enwikiSession=97juvgbur2g4fep52c9q0tl4dd038vks; centralauth_Session=d5d697afafddf1edce736a8e083d64aa
if-none-match:"815568267/8dd72aa6-e1fd-11e7-a66d-f2eb3c4dd428"
referer:https://en.wikipedia.org/wiki/Sonic_the_Hedgehog
user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36
x-requested-with:XMLHttpRequest

Response:
access-control-allow-headers:accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding
access-control-allow-methods:GET,HEAD
access-control-allow-origin:*
access-control-expose-headers:etag
age:634
cache-control:s-maxage=1209600, max-age=300
content-encoding:gzip
content-location:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
content-security-policy:default-src 'none'; frame-ancestors 'none'
content-type:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
date:Tue, 09 Jan 2018 16:32:32 GMT
etag:"815568267/8dd72aa6-e1fd-11e7-a66d-f2eb3c4dd428"
referrer-policy:origin-when-cross-origin
server:restbase1007
status:304
strict-transport-security:max-age=106384710; includeSubDomains; preload
vary:Accept-Encoding
via:1.1 varnish-v4, 1.1 varnish-v4
x-analytics:WMF-Last-Access=09-Jan-2018;WMF-Last-Access-Global=09-Jan-2018;https=1
x-cache:cp1066 hit/4, cp1066 hit/2
x-cache-status:hit-front
x-client-ip:73.252.38.252
x-content-security-policy:default-src 'none'; frame-ancestors 'none'
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-request-id:36848c22-f559-11e7-b6bd-f764a5a3c585
x-varnish:348855286 338894841, 190043427 194758879
x-webkit-csp:default-src 'none'; frame-ancestors 'none'
x-xss-protection:1; mode=block
  1. Go offline and hover over the same link. The request still fails:
Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
Referrer Policy:origin-when-cross-origin

Request:
Provisional headers are shown
Accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
Referer:https://en.wikipedia.org/wiki/Sonic_the_Hedgehog
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36
X-Requested-With:XMLHttpRequest

The age keeps increasing and the page previews are never shown offline again. In summary, it seems that page previews are never shown for refreshed cached responses.

Browser affected:

  • Chromium v63.0.3239.84 on Ubuntu v17.10

Details

Related Gerrit Patches:
mediawiki/extensions/Popups : masterUpdate: show placeholder preview for more failures

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 9 2018, 4:41 PM
Niedzielski updated the task description. (Show Details)Jan 9 2018, 4:44 PM
Niedzielski updated the task description. (Show Details)
Niedzielski updated the task description. (Show Details)Jan 9 2018, 4:46 PM
Niedzielski updated the task description. (Show Details)
Niedzielski updated the task description. (Show Details)Jan 9 2018, 4:54 PM
Niedzielski updated the task description. (Show Details)Jan 9 2018, 4:56 PM
Niedzielski moved this task from Incoming to Needs Prioritization on the Readers-Web-Backlog board.
Niedzielski renamed this task from Previews not shown when cached response is refreshed to Cached page previews not shown when refreshed.Jan 10 2018, 1:11 PM
Niedzielski updated the task description. (Show Details)
Jdlrobson added a subscriber: Jdlrobson.

Guessing this is an issue with the http request?

Change 403722 had a related patch set uploaded (by Niedzielski; owner: Sniedzielski):
[mediawiki/extensions/Popups@master] Update: show placeholder preview for more failures

https://gerrit.wikimedia.org/r/403722

Change 403722 merged by jenkins-bot:
[mediawiki/extensions/Popups@master] Update: show placeholder preview for more failures

https://gerrit.wikimedia.org/r/403722

This seems to be independent of login as well as an issue on the RESTBase docs page:

  1. Go to https://en.wikipedia.org/api/rest_v1/#!/Page_content/get_page_summary_title
  2. Issue a request for "Orange" (works):
Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
Request Method:GET
Status Code:200 
Remote Address:208.80.154.224:443
Referrer Policy:origin-when-cross-origin

Request:
:authority:en.wikipedia.org
:method:GET
:path:/api/rest_v1/page/summary/Orange
:scheme:https
accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9
cookie:CP=H2; GeoIP=US:TN:Cordova:35.14:-89.77:v4; loginnotify_prevlogins=2017-4bna7w-5u9tr14qqj8vtxax4uf44099fgftf9g; VEE=wikitext; enwikiUserID=13204772; enwikiUserName=Niedzielski; centralauth_User=Niedzielski; centralauth_Token=1f3ae7c802862ac7849ccb8288230417; forceHTTPS=true; optin=beta; WMF-Last-Access=19-Jan-2018; WMF-Last-Access-Global=19-Jan-2018; enwikiSession=jhk74roldib9ksjs9qfejg78gas8rirn; centralauth_Session=36d8332c4448b36e8606b4c6edb322a6
referer:https://en.wikipedia.org/api/rest_v1/
user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36

Response:
accept-ranges:bytes
access-control-allow-headers:accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding
access-control-allow-methods:GET,HEAD
access-control-allow-origin:*
access-control-expose-headers:etag
age:0
cache-control:s-maxage=1209600, max-age=300
content-encoding:gzip
content-length:192
content-location:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
content-security-policy:default-src 'none'; frame-ancestors 'none'
content-type:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
date:Fri, 19 Jan 2018 15:30:26 GMT
etag:"814406686/6e0fc7b1-dc39-11e7-8bcf-05d54beef9fc"
referrer-policy:origin-when-cross-origin
server:restbase1007
status:200
strict-transport-security:max-age=106384710; includeSubDomains; preload
vary:Accept-Encoding
via:1.1 varnish-v4, 1.1 varnish-v4
x-analytics:WMF-Last-Access=19-Jan-2018;WMF-Last-Access-Global=19-Jan-2018;https=1
x-cache:cp1068 miss, cp1053 miss
x-cache-status:miss
x-client-ip:73.252.38.252
x-content-security-policy:default-src 'none'; frame-ancestors 'none'
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-request-id:ac4e7433-fd2d-11e7-ae36-c121ff3b1d4a
x-varnish:527281540, 335548617
x-webkit-csp:default-src 'none'; frame-ancestors 'none'
x-xss-protection:1; mode=block
  1. Go offline and issue the request again (works):
Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
Request Method:GET
Status Code:200  (from disk cache)
Remote Address:208.80.154.224:443
Referrer Policy:origin-when-cross-origin

Request:
Provisional headers are shown
Accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
Referer:https://en.wikipedia.org/api/rest_v1/
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36

Response:
accept-ranges:bytes
access-control-allow-headers:accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding
access-control-allow-methods:GET,HEAD
access-control-allow-origin:*
access-control-expose-headers:etag
age:0
cache-control:s-maxage=1209600, max-age=300
content-encoding:gzip
content-length:192
content-location:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
content-security-policy:default-src 'none'; frame-ancestors 'none'
content-type:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
date:Fri, 19 Jan 2018 15:30:26 GMT
etag:"814406686/6e0fc7b1-dc39-11e7-8bcf-05d54beef9fc"
referrer-policy:origin-when-cross-origin
server:restbase1007
status:200
vary:Accept-Encoding
via:1.1 varnish-v4, 1.1 varnish-v4
x-analytics:WMF-Last-Access=19-Jan-2018;WMF-Last-Access-Global=19-Jan-2018;https=1
x-cache:cp1068 miss, cp1053 miss
x-cache-status:miss
x-client-ip:73.252.38.252
x-content-security-policy:default-src 'none'; frame-ancestors 'none'
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-request-id:ac4e7433-fd2d-11e7-ae36-c121ff3b1d4a
x-varnish:527281540, 335548617
x-webkit-csp:default-src 'none'; frame-ancestors 'none'
x-xss-protection:1; mode=block
  1. Wait 5 minutes and issue the request again (fails as expected):
Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
Referrer Policy:origin-when-cross-origin

Request:
Provisional headers are shown
Accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
Referer:https://en.wikipedia.org/api/rest_v1/
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36
  1. Go online and issue the request again (works):
Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
Request Method:GET
Status Code:200 
Remote Address:208.80.154.224:443
Referrer Policy:origin-when-cross-origin

Request:
:authority:en.wikipedia.org
:method:GET
:path:/api/rest_v1/page/summary/Orange
:scheme:https
accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9
cookie:CP=H2; GeoIP=US:TN:Cordova:35.14:-89.77:v4; loginnotify_prevlogins=2017-4bna7w-5u9tr14qqj8vtxax4uf44099fgftf9g; VEE=wikitext; enwikiUserID=13204772; enwikiUserName=Niedzielski; centralauth_User=Niedzielski; centralauth_Token=1f3ae7c802862ac7849ccb8288230417; forceHTTPS=true; optin=beta; WMF-Last-Access=19-Jan-2018; WMF-Last-Access-Global=19-Jan-2018; enwikiSession=jhk74roldib9ksjs9qfejg78gas8rirn; centralauth_Session=36d8332c4448b36e8606b4c6edb322a6
if-none-match:"814406686/6e0fc7b1-dc39-11e7-8bcf-05d54beef9fc"
referer:https://en.wikipedia.org/api/rest_v1/
user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36

Response:
accept-ranges:bytes
access-control-allow-headers:accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding
access-control-allow-methods:GET,HEAD
access-control-allow-origin:*
access-control-expose-headers:etag
age:388
cache-control:s-maxage=1209600, max-age=300
content-encoding:gzip
content-length:192
content-location:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
content-security-policy:default-src 'none'; frame-ancestors 'none'
content-type:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
date:Fri, 19 Jan 2018 15:36:54 GMT
etag:"814406686/6e0fc7b1-dc39-11e7-8bcf-05d54beef9fc"
referrer-policy:origin-when-cross-origin
server:restbase1007
status:200
strict-transport-security:max-age=106384710; includeSubDomains; preload
vary:Accept-Encoding
via:1.1 varnish-v4, 1.1 varnish-v4
x-analytics:WMF-Last-Access=19-Jan-2018;WMF-Last-Access-Global=19-Jan-2018;https=1
x-cache:cp1068 hit/1, cp1053 miss
x-cache-status:hit-local
x-client-ip:73.252.38.252
x-content-security-policy:default-src 'none'; frame-ancestors 'none'
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-request-id:ac4e7433-fd2d-11e7-ae36-c121ff3b1d4a
x-varnish:520145599 527281541, 332722153
x-webkit-csp:default-src 'none'; frame-ancestors 'none'
x-xss-protection:1; mode=block
  1. Go offline and issue the request again (fails unexpectedly):
Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
Referrer Policy:origin-when-cross-origin

Request:
Accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
Referer:https://en.wikipedia.org/api/rest_v1/
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36

I haven't tracked it down for other cases but there seems to be inconsistent behavior with different titles like "Paprika" immediately failed to cache for me but "Lime" has the same behavior as above. I also see funkiness with different page redirects.

ovasileva triaged this task as High priority.Jan 30 2018, 8:52 PM
ovasileva added a subscriber: ovasileva.

do we know if this is happening on all projects?

We'll need some input from services around this. @Pchelolo any ideas?

@Jdlrobson We're on an offsite so if it can wait till tomorrow I'll have a look tomorrow.

hm.. I was able to reproduce this in Safari as well. I'm not sure what could be special about these articles, we set the exact same cache-control headers for all the responses, so I don't think this is a RESTBase issue. Will dig a bit more

I think I might have found a correlation here - we (Varnish actually) set the age header of the response to the number of seconds the entry has been in Varnish. Whenever the age is smaller the max-age the browser caches the response. Whenever the age is larger than max-age the browser doesn't use the cached response even though that's not the age from the disc cache.

Pchelolo added a comment.EditedFeb 2 2018, 8:23 PM

Yep, just confirmed that locally - whenever the age header is larget the max-age header local browser HTTP caching is not happening. This is weird cause the age header is defined for the proxy caches, not the client-side cache.

This is pretty serious cause the vast majority of requests for summary age served by Varnish with large values of the age header, meaning the client-side caching almost never happens in reality.

This comment was removed by Pchelolo.

Looks like services would be best place to fix this?

phuedx added a subscriber: phuedx.Feb 5 2018, 6:51 PM
Niedzielski added a subscriber: BBlack.

We discussed this offline and per @Pchelolo's investigation, it seems to be an issue with the age header set by Varnish. @BBlack, can you help route this issue to the appropriate cache expert?

Volans edited projects, added Operations, Traffic; removed SRE-tools.Feb 7 2018, 7:16 PM

I've done a little bit more research here and Varnish docs actually confirm that age header can effectively disallow the client-side caching, see http://book.varnish-software.com/3.0/HTTP.html#age

One of the solutions I've found on the internet for this is to rename age to x-age in cases where the response cache-control has higher s-max-age the max-age to avoid disabling client-side caching for the vast majority of responses. @BBlack Do you think we could do something like that?

BBlack added a comment.Feb 8 2018, 1:16 AM

I don't know if that sounds like quite the right answer, I think this needs more thinking/info about what behaviors we're trying to actually accomplish here. I'm guessing the reason RB is emitting a large s-maxage combined with a short max-age is that we can purge the varnishes, but we can't purge the clients. This is a reasonable line of thinking, but without explicit VCL hacks to support it, we're not technically even doing this part right (as we're exposing s-maxage beyond our border of purging control, where in theory a 3rd party [tls-intercepting!] shared cache could hold the item for the whole s-maxage without purge control).

For normal article pageviews via MediaWiki, there is a similar issue, and it's solved by having custom VCL disallow external caching entirely, so that the only caching happens within our purge control, but this doesn't consider an offline use-case.

I think to really comprehend the right fix here, I'd need to rewind a little and figure out what we're really trying to accomplish, in terms of purging, cache lifetimes, allowable staleness for live clients, offline usage/staleness, etc.

I think to really comprehend the right fix here, I'd need to rewind a little and figure out what we're really trying to accomplish, in terms of purging, cache lifetimes, allowable staleness for live clients, offline usage/staleness, etc.

The max-age: 300 part of the header was introduced to minimise incidental HTTP requests that make it to our edge triggered by the user casually moving their mouse over the page. 300 was chosen as it matched the value used for the MediaWiki API version of Page Previews. See T161284: Minimise incidental HTTP requests caused by Page Previews for additional context and discussion.

For normal article pageviews via MediaWiki, there is a similar issue, and it's solved by having custom VCL disallow external caching entirely, so that the only caching happens within our purge control, but this doesn't consider an offline use-case.

So we'd rewrite the header to something like Cache-Control: private, must-revalidate, max-age: 300 at the edge (per the suggestion in T161284#3142452)?

Well, the above solution would still leave your with your short-age problem, if you didn't also zero out the Age.

Do we want to allow stale content in the UA's cache here, for up to 5 minutes past the expiry? There are also ways we can structure this so that it's never stale, but it's a bit more complicated.

Is offline viewing of previously-seen previews expected to fail and stop showing the (possibly stale) cached contents after exactly 5 minutes offline?

ema moved this task from Triage to Caching on the Traffic board.Feb 14 2018, 11:00 AM

Do we want to allow stale content in the UA's cache here, for up to 5 minutes past the expiry?

Yes, although, reading back over T161284: Minimise incidental HTTP requests caused by Page Previews, it looks like 3 minutes was the suggestion from Performance-Team.

Is offline viewing of previously-seen previews expected to fail and stop showing the (possibly stale) cached contents after exactly 5 minutes offline?

We didn't consider this while designing the system, nor when discussing minimising incidental traffic. I think that if we were to allow possibly previews to be shown when the user has been offline for longer than 5 minutes, then we should also indicate to the user that we're doing so /cc @Nirzar

@BBlack, @phuedx sorry to be a bother. This seems like an important issue as we're trying to rollout page previews to prod this quarter. Is there any way we can move forward on this issue? Do need input from design or performance? /cc @ovasileva

I tested a few more endpoints just using the documentation site and here's what I saw:

EndpointTesting redirect?max-age is nonzeroage can be nonzeroage resets after max-age expiresmust-revalidate is omittedCacheable on the client?Notes
/page/title/{title}
/page/html/{title}
/page/summary/{title}Almost noAge never resets so responses cached on the backend can't be cached on the client (this bug)
Redirect behaves differently from endpoint and can't be cached
/page/related/{title}
/page/pdf/{title}
Redirect behaves differently from the endpoint and can't be cached
/page/mobile-sections/{title}

Conclusions:

  • Redirects behave consistently with respect to each other but not always with respect to their endpoint.
  • When either this issue or T156829 is in effect it prevents all clients from using responses in periods of low connectivity. For this reason, and conserving user and Wikipedia data, these bugs seem very high priority to me.
238482n375 set Security to Software security bug.Jun 15 2018, 8:06 AM
238482n375 added a project: Security.
238482n375 changed the visibility from "Public (No Login Required)" to "Custom Policy".
238482n375 added a subscriber: 238482n375.
This comment was removed by Vgutierrez.
Vgutierrez changed the visibility from "Custom Policy" to "Public (No Login Required)".
Vgutierrez removed subscribers: Vgutierrez, 238482n375.
Vgutierrez added a subscriber: Vgutierrez.
Restricted Application added a project: Security. · View Herald TranscriptJun 15 2018, 10:35 AM