This task is about making the previous work from T409577 a permanent dataset in the Data Lake. It includes:
| Title | Reference | Author | Source Branch | Dest Branch | |
|---|---|---|---|---|---|
| Shift the start_date of the ja3n_ua_hourly DAG back so that we include an... | repos/data-engineering/airflow-dags!1852 | mforns | shift-ja3n-ua-hourly-start-date | main | |
| Add ja3n_ua_hourly DAG | repos/data-engineering/airflow-dags!1842 | mforns | add-ja3n-ua-hourly-dag | main |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | mforns | T407893 Technical work for SDS1.3.7 Incorporate Edge Signal | |||
| Resolved | mforns | T409584 Productionize JA3N-UA table to improve bot detection |
Change #1212214 had a related patch set uploaded (by Mforns; author: Mforns):
[analytics/refinery@master] Add JA3N User-Agent queries
mforns opened https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/1842
Add ja3n_ua_hourly DAG
Change #1212626 had a related patch set uploaded (by Mforns; author: Mforns):
[operations/puppet@production] analytics::refinery::job::data_purge: Add drop-ja3n-ua-hourly job
Change #1212214 merged by Mforns:
[analytics/refinery@master] Add JA3N User-Agent queries
mforns merged https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/1842
Add ja3n_ua_hourly DAG
Change #1213488 had a related patch set uploaded (by Mforns; author: Mforns):
[analytics/refinery@master] Add user_agent_map to the ja3n_ua_hourly table
Change #1213488 merged by Mforns:
[analytics/refinery@master] Add user_agent_map to the ja3n_ua_hourly table
Change #1213522 had a related patch set uploaded (by Mforns; author: Mforns):
[analytics/refinery@master] Fix issue in ja3n_ua_hourly query
Change #1213522 merged by Mforns:
[analytics/refinery@master] Fix issue in ja3n_ua_hourly query
Change #1213572 had a related patch set uploaded (by Mforns; author: Mforns):
[analytics/refinery@master] Modify ja3n_ua_hourly queries to migrate the table to Iceberg
Change #1213572 merged by Mforns:
[analytics/refinery@master] Modify ja3n_ua_hourly queries to migrate the table to Iceberg
mforns opened https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/1852
Shift the start_date of the ja3n_ua_hourly DAG back so that we include an...
mforns merged https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/1852
Shift the start_date of the ja3n_ua_hourly DAG back so that we include an...