When we have the results from the sub-ref related scraper runs we'll need to update the google sheet with the data.
See e.g. 🖹 the old sheet
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T396752 [Epic] Metrics for sub-referencing | |||
| Open | None | T398277 [Epic] Scraping data dumps and scraper enhancements | |||
| Open | None | T396731 Scraper: Build a new dashboard for the updated scraper data | |||
| Open | None | T410719 Persist scraper outputs to Hive | |||
| Resolved | None | T412227 Rate limit EventGate output to 500 events/second | |||
| Open | None | T413567 Discussion: scraper outputs vs. event sanitization | |||
| Open | None | T412019 [Epic] Schedule scraper and aggregations as an Airflow job | |||
| Open | WMDE-Fisch | T413954 Set up Hive aggregation table and populate with sample data | |||
| Open | None | T414803 Write an Airflow sensor to detect Enterprise Snapshots | |||
| Open | None | T414804 Produce conda-dist package for scraper | |||
| Open | None | T414809 Write an Airflow sensor for scraper page summaries being fully imported to Hive | |||
| Open | None | T413971 Create a Superset "dataset" exposing the scraper per-wiki results | |||
| Duplicate | WMDE-Fisch | T414057 Deduplicate page data that got logged multiple times during one run |