Maniphest T328365

SPIKE: Adapt our pipelines to codfw switch
Closed, ResolvedPublic3 Estimated Story Points
Actions

Assigned To

Authored By

	Antoine_Quhen
	Jan 30 2023, 7:56 PM

Tags

Referenced Files

None

Subscribers

Description

The CODFW DC has been producing events for a while but very few. Some of our pipelines may not be ready if a significant part of our traffic passes through CODFW. Example:
https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/blob/main/analytics/dags/wikidata/item_page_link/wikidata_item_page_link_weekly_dag.py#L100

https://phabricator.wikimedia.org/T327920

Spike outcomes:

Related Objects

Mentioned Here: T327920: March 2023 Datacenter Switchover

Event Timeline

Antoine_Quhen created this task.Jan 30 2023, 7:56 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 30 2023, 7:56 PM

• EChetty moved this task from Backlog to To be discussed /To be estimated on the Data Pipelines board.Jan 31 2023, 4:15 PM

Antoine_Quhen renamed this task from Adapt our pipelines to codfw switch to Adapt our pipelines to codfw switch (estimation).Feb 6 2023, 3:16 PM

• EChetty renamed this task from Adapt our pipelines to codfw switch (estimation) to SPIKE: Adapt our pipelines to codfw switch .Feb 6 2023, 4:19 PM

• EChetty triaged this task as High priority.

• EChetty updated the task description. (Show Details)

• EChetty set the point value for this task to 3.

• EChetty moved this task from To be discussed /To be estimated to Sprint 08 on the Data Pipelines board.Feb 7 2023, 12:29 PM

• EChetty edited projects, added Data Pipelines (Sprint 08); removed Data Pipelines.

• EChetty moved this task from Ready to Next Up on the Data Pipelines (Sprint 08) board.

JArguello-WMF edited projects, added Data Pipelines (Sprint 11); removed Data Pipelines (Sprint 08).Feb 20 2023, 5:18 PM

JArguello-WMF moved this task from Next Up to Next Up on the Data Pipelines (Sprint 11) board.Feb 20 2023, 5:18 PM

JArguello-WMF assigned this task to mforns.Feb 21 2023, 5:15 PM

JArguello-WMF moved this task from Next Up to In Progress on the Data Pipelines (Sprint 11) board.Feb 23 2023, 5:07 PM

lbowmaker moved this task from In Progress to In Review on the Data Pipelines (Sprint 11) board.Feb 27 2023, 5:03 PM

mforns moved this task from In Review to Blocked/Paused on the Data Pipelines (Sprint 11) board.Mar 1 2023, 11:06 PM

JArguello-WMF edited projects, added Data Pipelines; removed Data Pipelines (Sprint 11).Mar 2 2023, 5:47 PM

JArguello-WMF added a project: Data Engineering and Event Platform Team.Jun 30 2023, 5:54 PM

lbowmaker edited projects, added Data-Engineering; removed Data Engineering and Event Platform Team.Nov 10 2023, 2:49 PM

Automated now