Create intermediate dataset: pageview + is_redirect_to_pageview with actor information for easier computation of unique data (so we do not read a whole month of webrequest).
This dataset should also be usefull for other queries, as it would contain an actor-fingerprint id, and most fields from webrequest that currently are removed from pageview.
How do we call it?
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Correct pageview_actor_hourly bug | analytics/refinery | master | +8 -2 | |
| Add pageview_actor_hourly table and oozie job | analytics/refinery | master | +648 -0 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T138207 [Open question] Improve bot identification at scale | |||
| Resolved | None | T238357 Label high volume bot spikes in pageview data as automated traffic | |||
| Resolved | JAllemandou | T250744 Unique devices, retrofit with bot detection code | |||
| Resolved | JAllemandou | T255467 Create intermediate dataset: pageview with actor information |
I'm struggling with the name: The table will contain pageview AND redirect-to-pageview, so it's not not only pageview. Plus, it's a lot more similar to webrequest than to pageview...
Maybe webrequest_pageview_actor, and storing table and job stuff close to webrequerst instead of pageview?
Change 606127 had a related patch set uploaded (by Joal; owner: Joal):
[analytics/refinery@master] [WIP] Add pageview_actor_hourly table and oozie job
Change 606127 merged by Joal:
[analytics/refinery@master] Add pageview_actor_hourly table and oozie job
Change 607719 had a related patch set uploaded (by Joal; owner: Joal):
[analytics/refinery@master] Correct pageview_actor_hourly bug
Change 607719 merged by Joal:
[analytics/refinery@master] Correct pageview_actor_hourly bug