Page MenuHomePhabricator

Add cache_status to pageview_actor table
Closed, DeclinedPublic

Description

Description

Understanding the relative cost of requests often comes up in PP3 (previously WE5) Scalable Systems for Responsible Use objective area (including API requests, media traffic etc.). To begin with, we would like to understand whether a request was served from cached content (cache- hit or int) or not (cache- miss or pass). Currently, we are not able to estimate that for pageviews. For example, to answer questions like, what percentage of automated pageviews are not non-cached content?

To able to answer such questions, it would be helpful to have cache_status field in wmf.pageview_actor table (passed down from webrequest). Possibly in wmf.pageview_hourly as well. There is no regular reporting planned on this, so having only at pageview_actor is enough for now, however, if it isn't a lot of additional work having it in pageview_hourly would be great as well.

cache_status vs. _normalised

  • We are only interested in hit, int, miss and pass. So if we can normalise the statuses (example) that'll be easier during aggregations (not a hard requirement though).

Acceptance Criteria

  • cache_status or cache_status_normalised field in wmf.pageview_actor
  • (optional) cache_status or cache_status_normalised field in wmf.pageview_hourly

Event Timeline

I think the webrequest v2 table from the KAPOW project will respond to this need: it'll be partitioned by query-type, allowing to query pageview-only data, and will have the cache_status field.

Ottomata subscribed.

This data should be in webrequest_v2.