This task needs to be split in smaller chunks
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T324484 [Migrate] Oozie jobs migration for Webrequest | |||
| Resolved | Antoine_Quhen | T327072 Java Prep for Webrequest Load | |||
| Resolved | Antoine_Quhen | T325266 Replace refinery-source Guava caches by Caffeine | |||
| Resolved | Antoine_Quhen | T327073 Write Airflow DAG to move the webrequest load job to airflow. |
Event Timeline
Comment Actions
o/
If the plan is to move webrequest refinement to Spark, are we considering doing it with Refine? If we do T314956: Declare webrequest as an Event Platform stream (not hard), then we could manage the webrequest ingestion in the same way that we do every other stream.