Page MenuHomePhabricator

Add user_agent raw string to pageview_actor and pageview_hourly
Closed, DeclinedPublic

Description

Description

In PP3 FY26-27: we have an upcomign use case to track activity for specific clients across the our traffic, including pageviews. To be able to do this, we need the user_agent raw string in pageview actor and pageview_hourly. Additionally, having the raw string will enable us to join the user_agents_info table being created as part of T430020.

Acceptance Criteria

  • user_agent raw string available in pageview_actor and pageview_hourly

Event Timeline

Can you please expand on the "track activity" part? I'd like to understand what we need to measure here so we can find the best table(s) to create and change.

It sounds like what you're looking for is: How many pageviews came from a particular client, whether that is a self-identified bot or assumed human, aggregated to hourly counts.

Is that it?

Can you please expand on the "track activity" part? I'd like to understand what we need to measure here so we can find the best table(s) to create and change.

It sounds like what you're looking for is: How many pageviews came from a particular client, whether that is a self-identified bot or assumed human, aggregated to hourly counts.

Is that it?

Sorry missed this somehow. Some background that might be helpful: as we regularly produce list of potential WME customers and also some high volume clients that keep appearing every month, we would like to track their overall activity across our traffic, including but not limited to: API, pageviews, media, dumps etc. This will basically help to create usage report to understand current usage and potential for WME negotiations. The first step is to start with User-Agents.

I don't think it'll be ok to add the raw user agent string to the pageview_hourly table: the data is kept indefinitely, and therefore we don't wish to store PII.
However the new webrequest v2 from KAPOW should provide useful partitioning, allowing to query pageviews only, and having the raw user_agent field.

Indeed, KC and I chatted about this afterwards, and that's probably going to be the plan. As I understand it, this ticket is obsolete, so I'll close it now. Feel free to reopen if I'm misunderstanding.