Cached page previews not shown when refreshed
Open, HighPublic
Actions

Assigned To

None

Authored By

	• Niedzielski
	Jan 9 2018, 4:41 PM

Description

Login to English Wikipedia and enable page previews.
Open Chromium Developer Tools and click the network tab.
Visit the Sonic the Hedgehog article.
Hover over the "Sonic Spinball" (or a preview not requested previously) link. This is the initial request / 200 response:

Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
Request Method:GET
Status Code:200 
Remote Address:208.80.154.224:443
Referrer Policy:origin-when-cross-origin

Request:
:authority:en.wikipedia.org
:method:GET
:path:/api/rest_v1/page/summary/Sonic_Spinball
:scheme:https
accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9
cookie:CP=H2; GeoIP=US:TN:Cordova:35.14:-89.77:v4; loginnotify_prevlogins=2017-4bna7w-5u9tr14qqj8vtxax4uf44099fgftf9g; VEE=wikitext; enwikiUserID=13204772; enwikiUserName=Niedzielski; centralauth_User=Niedzielski; centralauth_Token=1f3ae7c802862ac7849ccb8288230417; forceHTTPS=true; optin=beta; WMF-Last-Access=09-Jan-2018; WMF-Last-Access-Global=09-Jan-2018; enwikiSession=97juvgbur2g4fep52c9q0tl4dd038vks; centralauth_Session=d5d697afafddf1edce736a8e083d64aa
referer:https://en.wikipedia.org/wiki/Sonic_the_Hedgehog
user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36
x-requested-with:XMLHttpRequest

Response:
accept-ranges:bytes
access-control-allow-headers:accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding
access-control-allow-methods:GET,HEAD
access-control-allow-origin:*
access-control-expose-headers:etag
age:0
cache-control:s-maxage=1209600, max-age=300
content-encoding:gzip
content-length:778
content-location:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
content-security-policy:default-src 'none'; frame-ancestors 'none'
content-type:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
date:Tue, 09 Jan 2018 16:21:57 GMT
etag:"815568267/8dd72aa6-e1fd-11e7-a66d-f2eb3c4dd428"
referrer-policy:origin-when-cross-origin
server:restbase1007
status:200
strict-transport-security:max-age=106384710; includeSubDomains; preload
vary:Accept-Encoding
via:1.1 varnish-v4, 1.1 varnish-v4
x-analytics:WMF-Last-Access=09-Jan-2018;WMF-Last-Access-Global=09-Jan-2018;https=1
x-cache:cp1066 miss, cp1066 miss
x-cache-status:miss
x-client-ip:73.252.38.252
x-content-security-policy:default-src 'none'; frame-ancestors 'none'
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-request-id:36848c22-f559-11e7-b6bd-f764a5a3c585
x-varnish:338894840, 181105768
x-webkit-csp:default-src 'none'; frame-ancestors 'none'
x-xss-protection:1; mode=block

Move the cursor away to dismiss the preview.
Go offline and hover over the same link. The response is now served from cache:

Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
Request Method:GET
Status Code:200  (from disk cache)
Remote Address:208.80.154.224:443
Referrer Policy:origin-when-cross-origin

Request:
Provisional headers are shown
Accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
Referer:https://en.wikipedia.org/wiki/Sonic_the_Hedgehog
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36
X-Requested-With:XMLHttpRequest

Response:
accept-ranges:bytes
access-control-allow-headers:accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding
access-control-allow-methods:GET,HEAD
access-control-allow-origin:*
access-control-expose-headers:etag
age:0
cache-control:s-maxage=1209600, max-age=300
content-encoding:gzip
content-length:778
content-location:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
content-security-policy:default-src 'none'; frame-ancestors 'none'
content-type:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
date:Tue, 09 Jan 2018 16:21:57 GMT
etag:"815568267/8dd72aa6-e1fd-11e7-a66d-f2eb3c4dd428"
referrer-policy:origin-when-cross-origin
server:restbase1007
status:200
vary:Accept-Encoding
via:1.1 varnish-v4, 1.1 varnish-v4
x-analytics:WMF-Last-Access=09-Jan-2018;WMF-Last-Access-Global=09-Jan-2018;https=1
x-cache:cp1066 miss, cp1066 miss
x-cache-status:miss
x-client-ip:73.252.38.252
x-content-security-policy:default-src 'none'; frame-ancestors 'none'
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-request-id:36848c22-f559-11e7-b6bd-f764a5a3c585
x-varnish:338894840, 181105768
x-webkit-csp:default-src 'none'; frame-ancestors 'none'
x-xss-protection:1; mode=block

Move the cursor away to dismiss the preview.
Wait at least five minutes.
Go online and hover over the same link. The response is refreshed from the network as a 200:

Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
Request Method:GET
Status Code:200 
Remote Address:208.80.154.224:443
Referrer Policy:origin-when-cross-origin

Request:
:authority:en.wikipedia.org
:method:GET
:path:/api/rest_v1/page/summary/Sonic_Spinball
:scheme:https
accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9
cookie:CP=H2; GeoIP=US:TN:Cordova:35.14:-89.77:v4; loginnotify_prevlogins=2017-4bna7w-5u9tr14qqj8vtxax4uf44099fgftf9g; VEE=wikitext; enwikiUserID=13204772; enwikiUserName=Niedzielski; centralauth_User=Niedzielski; centralauth_Token=1f3ae7c802862ac7849ccb8288230417; forceHTTPS=true; optin=beta; WMF-Last-Access=09-Jan-2018; WMF-Last-Access-Global=09-Jan-2018; enwikiSession=97juvgbur2g4fep52c9q0tl4dd038vks; centralauth_Session=d5d697afafddf1edce736a8e083d64aa
if-none-match:"815568267/8dd72aa6-e1fd-11e7-a66d-f2eb3c4dd428"
referer:https://en.wikipedia.org/wiki/Sonic_the_Hedgehog
user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36
x-requested-with:XMLHttpRequest

Response:
accept-ranges:bytes
access-control-allow-headers:accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding
access-control-allow-methods:GET,HEAD
access-control-allow-origin:*
access-control-expose-headers:etag
age:470
cache-control:s-maxage=1209600, max-age=300
content-encoding:gzip
content-length:778
content-location:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
content-security-policy:default-src 'none'; frame-ancestors 'none'
content-type:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
date:Tue, 09 Jan 2018 16:29:47 GMT
etag:"815568267/8dd72aa6-e1fd-11e7-a66d-f2eb3c4dd428"
referrer-policy:origin-when-cross-origin
server:restbase1007
status:200
strict-transport-security:max-age=106384710; includeSubDomains; preload
vary:Accept-Encoding
via:1.1 varnish-v4, 1.1 varnish-v4
x-analytics:WMF-Last-Access=09-Jan-2018;WMF-Last-Access-Global=09-Jan-2018;https=1
x-cache:cp1066 hit/3, cp1066 miss
x-cache-status:hit-local
x-client-ip:73.252.38.252
x-content-security-policy:default-src 'none'; frame-ancestors 'none'
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-request-id:36848c22-f559-11e7-b6bd-f764a5a3c585
x-varnish:344684204 338894841, 198281743
x-webkit-csp:default-src 'none'; frame-ancestors 'none'
x-xss-protection:1; mode=block

Go offline and hover over the same link. The request now fails:

Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
Referrer Policy:origin-when-cross-origin

Request:
Provisional headers are shown
Accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
Referer:https://en.wikipedia.org/wiki/Sonic_the_Hedgehog
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36
X-Requested-With:XMLHttpRequest

Go online. If you issue more requests at this point, the responses will all be 200s.
Refresh the page.
Hover over the same link. The response is now a 304:

Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
Request Method:GET
Status Code:304 
Remote Address:208.80.154.224:443
Referrer Policy:origin-when-cross-origin

Request:
:authority:en.wikipedia.org
:method:GET
:path:/api/rest_v1/page/summary/Sonic_Spinball
:scheme:https
accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9
cookie:CP=H2; GeoIP=US:TN:Cordova:35.14:-89.77:v4; loginnotify_prevlogins=2017-4bna7w-5u9tr14qqj8vtxax4uf44099fgftf9g; VEE=wikitext; enwikiUserID=13204772; enwikiUserName=Niedzielski; centralauth_User=Niedzielski; centralauth_Token=1f3ae7c802862ac7849ccb8288230417; forceHTTPS=true; optin=beta; WMF-Last-Access=09-Jan-2018; WMF-Last-Access-Global=09-Jan-2018; enwikiSession=97juvgbur2g4fep52c9q0tl4dd038vks; centralauth_Session=d5d697afafddf1edce736a8e083d64aa
if-none-match:"815568267/8dd72aa6-e1fd-11e7-a66d-f2eb3c4dd428"
referer:https://en.wikipedia.org/wiki/Sonic_the_Hedgehog
user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36
x-requested-with:XMLHttpRequest

Response:
access-control-allow-headers:accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding
access-control-allow-methods:GET,HEAD
access-control-allow-origin:*
access-control-expose-headers:etag
age:634
cache-control:s-maxage=1209600, max-age=300
content-encoding:gzip
content-location:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
content-security-policy:default-src 'none'; frame-ancestors 'none'
content-type:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
date:Tue, 09 Jan 2018 16:32:32 GMT
etag:"815568267/8dd72aa6-e1fd-11e7-a66d-f2eb3c4dd428"
referrer-policy:origin-when-cross-origin
server:restbase1007
status:304
strict-transport-security:max-age=106384710; includeSubDomains; preload
vary:Accept-Encoding
via:1.1 varnish-v4, 1.1 varnish-v4
x-analytics:WMF-Last-Access=09-Jan-2018;WMF-Last-Access-Global=09-Jan-2018;https=1
x-cache:cp1066 hit/4, cp1066 hit/2
x-cache-status:hit-front
x-client-ip:73.252.38.252
x-content-security-policy:default-src 'none'; frame-ancestors 'none'
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-request-id:36848c22-f559-11e7-b6bd-f764a5a3c585
x-varnish:348855286 338894841, 190043427 194758879
x-webkit-csp:default-src 'none'; frame-ancestors 'none'
x-xss-protection:1; mode=block

Go offline and hover over the same link. The request still fails:

Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Sonic_Spinball
Referrer Policy:origin-when-cross-origin

Request:
Provisional headers are shown
Accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
Referer:https://en.wikipedia.org/wiki/Sonic_the_Hedgehog
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36
X-Requested-With:XMLHttpRequest

The age keeps increasing and the page previews are never shown offline again. In summary, it seems that page previews are never shown for refreshed cached responses.

Browser affected:

Chromium v63.0.3239.84 on Ubuntu v17.10

Details

	Subject	Repo	Branch	Lines +/-
	Update: show placeholder preview for more failures	mediawiki/extensions/Popups	master	+73 -26

Customize query in gerrit

Related Objects

Mentioned In: T193055: [Spike, 4hrs] Should we abort in flight page previews requests?
T116840: Cached REST end point for imageinfo requests
T154635: [EPIC] Deploy page previews to English and German Wikipedia
Mentioned Here: T156829: Remove must-revalidate cache-control header from mobile-sections-lead and mobile-sections-remaining responses
T161284: Minimise incidental HTTP requests caused by Page Previews

Event Timeline

• Niedzielski created this task.Jan 9 2018, 4:41 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 9 2018, 4:41 PM

• Niedzielski updated the task description. (Show Details)Jan 9 2018, 4:44 PM

• Niedzielski updated the task description. (Show Details)

• Niedzielski updated the task description. (Show Details)Jan 9 2018, 4:46 PM

• Niedzielski updated the task description. (Show Details)

• Niedzielski updated the task description. (Show Details)Jan 9 2018, 4:54 PM

• Niedzielski updated the task description. (Show Details)Jan 9 2018, 4:56 PM

• Niedzielski moved this task from Incoming to Needs Prioritization on the Web-Team-Backlog board.

• Niedzielski renamed this task from Previews not shown when cached response is refreshed to Cached page previews not shown when refreshed.Jan 10 2018, 1:11 PM

• Niedzielski updated the task description. (Show Details)

Guessing this is an issue with the http request?

Change 403722 had a related patch set uploaded (by Niedzielski; owner: Sniedzielski):
[mediawiki/extensions/Popups@master] Update: show placeholder preview for more failures

https://gerrit.wikimedia.org/r/403722

gerritbot added a project: Patch-For-Review.Jan 11 2018, 6:59 PM

• Niedzielski removed a project: Patch-For-Review.Jan 17 2018, 3:13 PM

Change 403722 merged by jenkins-bot:
[mediawiki/extensions/Popups@master] Update: show placeholder preview for more failures

https://gerrit.wikimedia.org/r/403722

ReleaseTaggerBot added a project: MW-1.31-release-notes (WMF-deploy-2018-02-06 (1.31.0-wmf.20)).Jan 18 2018, 12:00 AM

• Niedzielski removed a project: MW-1.31-release-notes (WMF-deploy-2018-02-06 (1.31.0-wmf.20)).Jan 18 2018, 12:14 AM

This seems to be independent of login as well as an issue on the RESTBase docs page:

Go to https://en.wikipedia.org/api/rest_v1/#!/Page_content/get_page_summary_title
Issue a request for "Orange" (works):

Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
Request Method:GET
Status Code:200 
Remote Address:208.80.154.224:443
Referrer Policy:origin-when-cross-origin

Request:
:authority:en.wikipedia.org
:method:GET
:path:/api/rest_v1/page/summary/Orange
:scheme:https
accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9
cookie:CP=H2; GeoIP=US:TN:Cordova:35.14:-89.77:v4; loginnotify_prevlogins=2017-4bna7w-5u9tr14qqj8vtxax4uf44099fgftf9g; VEE=wikitext; enwikiUserID=13204772; enwikiUserName=Niedzielski; centralauth_User=Niedzielski; centralauth_Token=1f3ae7c802862ac7849ccb8288230417; forceHTTPS=true; optin=beta; WMF-Last-Access=19-Jan-2018; WMF-Last-Access-Global=19-Jan-2018; enwikiSession=jhk74roldib9ksjs9qfejg78gas8rirn; centralauth_Session=36d8332c4448b36e8606b4c6edb322a6
referer:https://en.wikipedia.org/api/rest_v1/
user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36

Response:
accept-ranges:bytes
access-control-allow-headers:accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding
access-control-allow-methods:GET,HEAD
access-control-allow-origin:*
access-control-expose-headers:etag
age:0
cache-control:s-maxage=1209600, max-age=300
content-encoding:gzip
content-length:192
content-location:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
content-security-policy:default-src 'none'; frame-ancestors 'none'
content-type:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
date:Fri, 19 Jan 2018 15:30:26 GMT
etag:"814406686/6e0fc7b1-dc39-11e7-8bcf-05d54beef9fc"
referrer-policy:origin-when-cross-origin
server:restbase1007
status:200
strict-transport-security:max-age=106384710; includeSubDomains; preload
vary:Accept-Encoding
via:1.1 varnish-v4, 1.1 varnish-v4
x-analytics:WMF-Last-Access=19-Jan-2018;WMF-Last-Access-Global=19-Jan-2018;https=1
x-cache:cp1068 miss, cp1053 miss
x-cache-status:miss
x-client-ip:73.252.38.252
x-content-security-policy:default-src 'none'; frame-ancestors 'none'
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-request-id:ac4e7433-fd2d-11e7-ae36-c121ff3b1d4a
x-varnish:527281540, 335548617
x-webkit-csp:default-src 'none'; frame-ancestors 'none'
x-xss-protection:1; mode=block

Go offline and issue the request again (works):

Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
Request Method:GET
Status Code:200  (from disk cache)
Remote Address:208.80.154.224:443
Referrer Policy:origin-when-cross-origin

Request:
Provisional headers are shown
Accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
Referer:https://en.wikipedia.org/api/rest_v1/
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36

Response:
accept-ranges:bytes
access-control-allow-headers:accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding
access-control-allow-methods:GET,HEAD
access-control-allow-origin:*
access-control-expose-headers:etag
age:0
cache-control:s-maxage=1209600, max-age=300
content-encoding:gzip
content-length:192
content-location:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
content-security-policy:default-src 'none'; frame-ancestors 'none'
content-type:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
date:Fri, 19 Jan 2018 15:30:26 GMT
etag:"814406686/6e0fc7b1-dc39-11e7-8bcf-05d54beef9fc"
referrer-policy:origin-when-cross-origin
server:restbase1007
status:200
vary:Accept-Encoding
via:1.1 varnish-v4, 1.1 varnish-v4
x-analytics:WMF-Last-Access=19-Jan-2018;WMF-Last-Access-Global=19-Jan-2018;https=1
x-cache:cp1068 miss, cp1053 miss
x-cache-status:miss
x-client-ip:73.252.38.252
x-content-security-policy:default-src 'none'; frame-ancestors 'none'
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-request-id:ac4e7433-fd2d-11e7-ae36-c121ff3b1d4a
x-varnish:527281540, 335548617
x-webkit-csp:default-src 'none'; frame-ancestors 'none'
x-xss-protection:1; mode=block

Wait 5 minutes and issue the request again (fails as expected):

Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
Referrer Policy:origin-when-cross-origin

Request:
Provisional headers are shown
Accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
Referer:https://en.wikipedia.org/api/rest_v1/
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36

Go online and issue the request again (works):

Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
Request Method:GET
Status Code:200 
Remote Address:208.80.154.224:443
Referrer Policy:origin-when-cross-origin

Request:
:authority:en.wikipedia.org
:method:GET
:path:/api/rest_v1/page/summary/Orange
:scheme:https
accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9
cookie:CP=H2; GeoIP=US:TN:Cordova:35.14:-89.77:v4; loginnotify_prevlogins=2017-4bna7w-5u9tr14qqj8vtxax4uf44099fgftf9g; VEE=wikitext; enwikiUserID=13204772; enwikiUserName=Niedzielski; centralauth_User=Niedzielski; centralauth_Token=1f3ae7c802862ac7849ccb8288230417; forceHTTPS=true; optin=beta; WMF-Last-Access=19-Jan-2018; WMF-Last-Access-Global=19-Jan-2018; enwikiSession=jhk74roldib9ksjs9qfejg78gas8rirn; centralauth_Session=36d8332c4448b36e8606b4c6edb322a6
if-none-match:"814406686/6e0fc7b1-dc39-11e7-8bcf-05d54beef9fc"
referer:https://en.wikipedia.org/api/rest_v1/
user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36

Response:
accept-ranges:bytes
access-control-allow-headers:accept, content-type, content-length, cache-control, accept-language, api-user-agent, if-match, if-modified-since, if-none-match, dnt, accept-encoding
access-control-allow-methods:GET,HEAD
access-control-allow-origin:*
access-control-expose-headers:etag
age:388
cache-control:s-maxage=1209600, max-age=300
content-encoding:gzip
content-length:192
content-location:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
content-security-policy:default-src 'none'; frame-ancestors 'none'
content-type:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
date:Fri, 19 Jan 2018 15:36:54 GMT
etag:"814406686/6e0fc7b1-dc39-11e7-8bcf-05d54beef9fc"
referrer-policy:origin-when-cross-origin
server:restbase1007
status:200
strict-transport-security:max-age=106384710; includeSubDomains; preload
vary:Accept-Encoding
via:1.1 varnish-v4, 1.1 varnish-v4
x-analytics:WMF-Last-Access=19-Jan-2018;WMF-Last-Access-Global=19-Jan-2018;https=1
x-cache:cp1068 hit/1, cp1053 miss
x-cache-status:hit-local
x-client-ip:73.252.38.252
x-content-security-policy:default-src 'none'; frame-ancestors 'none'
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-request-id:ac4e7433-fd2d-11e7-ae36-c121ff3b1d4a
x-varnish:520145599 527281541, 332722153
x-webkit-csp:default-src 'none'; frame-ancestors 'none'
x-xss-protection:1; mode=block

Go offline and issue the request again (fails unexpectedly):

Request URL:https://en.wikipedia.org/api/rest_v1/page/summary/Orange
Referrer Policy:origin-when-cross-origin

Request:
Accept:application/json; charset=utf-8; profile="https://www.mediawiki.org/wiki/Specs/Summary/1.2.0"
Referer:https://en.wikipedia.org/api/rest_v1/
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36

I haven't tracked it down for other cases but there seems to be inconsistent behavior with different titles like "Paprika" immediately failed to cache for me but "Lime" has the same behavior as above. I also see funkiness with different page redirects.

• Niedzielski added a project: RESTBase.Jan 19 2018, 3:46 PM

• Niedzielski added a parent task: T154635: [EPIC] Deploy page previews to English and German Wikipedia.Jan 30 2018, 8:26 PM

• Niedzielski mentioned this in T154635: [EPIC] Deploy page previews to English and German Wikipedia.

do we know if this is happening on all projects?

@ovasileva, I suspect so. I've repro'd it on several English Wikipedia articles and just now on German RESTBase (https://de.wikipedia.org/api/rest_v1/page/summary/2015 and https://de.wikipedia.org/api/rest_v1/page/summary/2017).

We'll need some input from services around this. @Pchelolo any ideas?

@Jdlrobson We're on an offsite so if it can wait till tomorrow I'll have a look tomorrow.

hm.. I was able to reproduce this in Safari as well. I'm not sure what could be special about these articles, we set the exact same cache-control headers for all the responses, so I don't think this is a RESTBase issue. Will dig a bit more

I think I might have found a correlation here - we (Varnish actually) set the age header of the response to the number of seconds the entry has been in Varnish. Whenever the age is smaller the max-age the browser caches the response. Whenever the age is larger than max-age the browser doesn't use the cached response even though that's not the age from the disc cache.

Yep, just confirmed that locally - whenever the age header is larget the max-age header local browser HTTP caching is not happening. This is weird cause the age header is defined for the proxy caches, not the client-side cache.

This is pretty serious cause the vast majority of requests for summary age served by Varnish with large values of the age header, meaning the client-side caching almost never happens in reality.

• Pchelolo added a comment.Feb 2 2018, 8:46 PM

This comment was removed by • Pchelolo.

Looks like services would be best place to fix this?

phuedx subscribed.Feb 5 2018, 6:51 PM

Jdlrobson moved this task from Untriaged to Untag on the Web-Team-Backlog (Tracking) board.Feb 7 2018, 5:50 PM

We discussed this offline and per @Pchelolo's investigation, it seems to be an issue with the age header set by Varnish. @BBlack, can you help route this issue to the appropriate cache expert?

Volans edited projects, added SRE, Traffic; removed SRE-tools.Feb 7 2018, 7:16 PM

• Mholloway subscribed.Feb 7 2018, 7:57 PM

I've done a little bit more research here and Varnish docs actually confirm that age header can effectively disallow the client-side caching, see http://book.varnish-software.com/3.0/HTTP.html#age

One of the solutions I've found on the internet for this is to rename age to x-age in cases where the response cache-control has higher s-max-age the max-age to avoid disabling client-side caching for the vast majority of responses. @BBlack Do you think we could do something like that?

I don't know if that sounds like quite the right answer, I think this needs more thinking/info about what behaviors we're trying to actually accomplish here. I'm guessing the reason RB is emitting a large s-maxage combined with a short max-age is that we can purge the varnishes, but we can't purge the clients. This is a reasonable line of thinking, but without explicit VCL hacks to support it, we're not technically even doing this part right (as we're exposing s-maxage beyond our border of purging control, where in theory a 3rd party [tls-intercepting!] shared cache could hold the item for the whole s-maxage without purge control).

For normal article pageviews via MediaWiki, there is a similar issue, and it's solved by having custom VCL disallow external caching entirely, so that the only caching happens within our purge control, but this doesn't consider an offline use-case.

I think to really comprehend the right fix here, I'd need to rewind a little and figure out what we're really trying to accomplish, in terms of purging, cache lifetimes, allowable staleness for live clients, offline usage/staleness, etc.

In T184534#3954704, @BBlack wrote:

I think to really comprehend the right fix here, I'd need to rewind a little and figure out what we're really trying to accomplish, in terms of purging, cache lifetimes, allowable staleness for live clients, offline usage/staleness, etc.

The max-age: 300 part of the header was introduced to minimise incidental HTTP requests that make it to our edge triggered by the user casually moving their mouse over the page. 300 was chosen as it matched the value used for the MediaWiki API version of Page Previews. See T161284: Minimise incidental HTTP requests caused by Page Previews for additional context and discussion.

For normal article pageviews via MediaWiki, there is a similar issue, and it's solved by having custom VCL disallow external caching entirely, so that the only caching happens within our purge control, but this doesn't consider an offline use-case.

So we'd rewrite the header to something like Cache-Control: private, must-revalidate, max-age: 300 at the edge (per the suggestion in T161284#3142452)?

Well, the above solution would still leave your with your short-age problem, if you didn't also zero out the Age.

Do we want to allow stale content in the UA's cache here, for up to 5 minutes past the expiry? There are also ways we can structure this so that it's never stale, but it's a bit more complicated.

Is offline viewing of previously-seen previews expected to fail and stop showing the (possibly stale) cached contents after exactly 5 minutes offline?

• ema moved this task from Backlog to Caching on the Traffic board.Feb 14 2018, 11:00 AM

In T184534#3967535, @BBlack wrote:

Do we want to allow stale content in the UA's cache here, for up to 5 minutes past the expiry?

Yes, although, reading back over T161284: Minimise incidental HTTP requests caused by Page Previews, it looks like 3 minutes was the suggestion from Performance-Team.

Is offline viewing of previously-seen previews expected to fail and stop showing the (possibly stale) cached contents after exactly 5 minutes offline?

We didn't consider this while designing the system, nor when discussing minimising incidental traffic. I think that if we were to allow possibly previews to be shown when the user has been offline for longer than 5 minutes, then we should also indicate to the user that we're doing so /cc @Nirzar

@BBlack, @phuedx sorry to be a bother. This seems like an important issue as we're trying to rollout page previews to prod this quarter. Is there any way we can move forward on this issue? Do need input from design or performance? /cc @ovasileva

I tested a few more endpoints just using the documentation site and here's what I saw:

Endpoint	Testing redirect?	max-age is nonzero	age can be nonzero	age resets after max-age expires	must-revalidate is omitted	Cacheable on the client?	Notes
/page/title/{title}		✗	✗	✓	✗	✗
	✓	✗	✓	✗	✗	✗
/page/html/{title}		✗	✓	✗	✗	✗
	✓	✗	✓	✗	✗	✗
/page/summary/{title}		✓	✓	✗	✓	Almost no	Age never resets so responses cached on the backend can't be cached on the client (this bug)
	✓	✗	✓	✗	✗	✗	Redirect behaves differently from endpoint and can't be cached
/page/related/{title}		✗	✗	✓	✗	✗
	✓	✗	✓	✗	✗	✗
/page/pdf/{title}		✓	✗	✓	✓	✓
	✓	✗	✓	✗	✗	✗	Redirect behaves differently from the endpoint and can't be cached
/page/mobile-sections/{title}		✗	✓	✗	✗	✗
	✓	✗	✓	✗	✗	✗

Conclusions:

Redirects behave consistently with respect to each other but not always with respect to their endpoint.
When either this issue or T156829 is in effect it prevents all clients from using responses in periods of low connectivity. For this reason, and conserving user and Wikipedia data, these bugs seem very high priority to me.

ovasileva moved this task from Backlog to incoming on the Page-Previews board.Mar 15 2018, 3:56 PM

Jdlrobson removed a parent task: T154635: [EPIC] Deploy page previews to English and German Wikipedia.Mar 30 2018, 12:28 AM

• Niedzielski mentioned this in T116840: Cached REST end point for imageinfo requests.Jun 11 2018, 3:15 PM

• Niedzielski mentioned this in T193055: [Spike, 4hrs] Should we abort in flight page previews requests?.Jun 12 2018, 6:30 PM

• 238482n375 set Security to Software security bug.Jun 15 2018, 8:06 AM

• 238482n375 added a project: acl*security.

• 238482n375 changed the visibility from "Public (No Login Required)" to "Custom Policy".

• 238482n375 subscribed.

This comment was removed by Vgutierrez.

Vgutierrez removed a project: acl*security.Jun 15 2018, 10:34 AM

Vgutierrez changed the visibility from "Custom Policy" to "Public (No Login Required)".

Vgutierrez removed subscribers: Vgutierrez, • 238482n375.

Vgutierrez subscribed.

Restricted Application added a project: acl*security. · View Herald TranscriptJun 15 2018, 10:35 AM

Vgutierrez removed a project: acl*security.Jun 15 2018, 10:35 AM

• Jhernandez subscribed.Jun 18 2018, 8:53 AM

• Niedzielski awarded a token.Dec 3 2018, 5:26 PM

• mobrovac added a project: Platform Team Legacy (Watching / External).Dec 20 2018, 12:01 PM

• Phabricator_maintenance moved this task from Backlog to Acknowledged on the SRE board.Jan 26 2019, 9:41 PM

• Jhernandez unsubscribed.Apr 2 2020, 6:46 PM

Jdlrobson moved this task from incoming to Tracking on the Page-Previews board.Apr 14 2021, 6:49 PM

Jdlrobson edited projects, added Page-Previews (Tracking); removed Page-Previews.

Jdlrobson moved this task from Untag to Untriaged on the Web-Team-Backlog (Tracking) board.Sep 7 2021, 9:06 PM

Jdlrobson moved this task from Untriaged to Untag on the Web-Team-Backlog (Tracking) board.Sep 7 2021, 9:11 PM

BBlack moved this task from Caching to Icebox-Temp on the Traffic board.Oct 8 2021, 5:26 PM

The swap of Traffic for Traffic-Icebox in this ticket's set of tags was based on a bulk action for all tickets that aren't are neither part of our current planned work nor clearly a recent, higher-priority emergent issue. This is simply one step in a larger task cleanup effort. Further triage of these tickets (and especially, organizing future potential project ideas from them into a new medium) will occur afterwards! For more detail, have a look at the extended explanation on the main page of Traffic-Icebox . Thank you!

LGoto removed a project: Web-Team-Backlog (Tracking).Nov 9 2021, 4:23 PM

Cached page previews not shown when refreshedOpen, HighPublicActions

Description

Details

Related Objects

Event Timeline

Cached page previews not shown when refreshed
Open, HighPublic
Actions