Page MenuHomePhabricator

AQS pageview default caching is one day
Closed, ResolvedPublic


Today Joseph realized that the pageview tool in cloud wasn't showing any bump for the Maradona pageviews, and eventually we realized that the AQS API results were cached and not updated.

What I did was explicitly forcing a purge of the page via mwmaint1002, but I believe that we should think about a longer term fix to avoid this from re-happening. My understanding is that we don't set any caching headers, so we get the default 1 day in Varnish/ATS:

curl -I
HTTP/2 200
cache-control: s-maxage=86400, max-age=86400    <=====================
content-type: application/json; charset=utf-8
server: restbase1020
date: Thu, 26 Nov 2020 09:07:57 GMT

Should we move to something like 4/6/12 hours instead?

Event Timeline

elukey created this task.Nov 26 2020, 9:25 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 26 2020, 9:25 AM

Thanks @elukey for this ticket.
Given that the loading of the data usually finishes before 2AM (UTC) of the current day for the previous day, high-traffic pages would most probably have been requested before being loaded, leading to not up-to-date values being cached. The faster we purge after this, the better - I think that 4 hour would be great, I'd still be happy with 6, but 12 seems too much (and the current is definitely not good :)

fdans added a comment.Nov 30 2020, 4:58 PM

agreed on 4 during triage

fdans assigned this task to razzi.Dec 10 2020, 5:44 PM
fdans triaged this task as High priority.
fdans moved this task from Incoming to Operational Excellence on the Analytics board.

Change 647756 had a related patch set uploaded (by Razzi; owner: Razzi):
[analytics/aqs@master] Make aqs cache for 4 hours

Change 649884 had a related patch set uploaded (by Fdans; owner: Fdans):
[analytics/aqs@master] AQS: add configuration for timeout and max retries to Druit requests

Change 647756 merged by Fdans:
[analytics/aqs@master] Make aqs cache for 4 hours

Change 649884 merged by Fdans:
[analytics/aqs@master] AQS: add configuration for timeout to Druid requests

Mentioned in SAL (#wikimedia-operations) [2021-01-05T20:48:14Z] <razzi@deploy1001> Started deploy [analytics/aqs/deploy@5d05f83]: Configure http request timeout and caching for T268809

Mentioned in SAL (#wikimedia-operations) [2021-01-05T20:53:03Z] <razzi@deploy1001> Finished deploy [analytics/aqs/deploy@5d05f83]: Configure http request timeout and caching for T268809 (duration: 04m 48s)

razzi moved this task from Next Up to Done on the Analytics-Kanban board.Thu, Jan 21, 5:52 PM
fdans closed this task as Resolved.Mon, Jan 25, 7:00 PM