Page MenuHomePhabricator
Create Task
Maniphest T393560

Data pipeline to load cx_corpora to Data Lake, at wmf_product
Closed, ResolvedPublic
Actions

Description

https://www.mediawiki.org/wiki/Extension:ContentTranslation/cx_corpora_table

to be loaded to Data Lake at

wmf_product.cx_corpora

Details

Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
Update query commit IDs - translations and corporarepos/product-analytics/content-translation-airflow-jobs!5kcvelagacx-translationsmain
Customize query in GitLab

Related Objects
Search...

Event Timeline

KCVelaga_WMF created this task.May 7 2025, 8:29 AM
KCVelaga_WMF edited parent tasks, added: T393561: Setup pipelines to load CX extension tables into Data Lake, at wmf_product; removed: T393559: Data pipeline to load cx_translations to Data Lake, at wmf_product.May 7 2025, 8:34 AM
CodeReviewBot added a project: Patch-For-Review.May 7 2025, 10:35 AM

kcvelaga opened https://gitlab.wikimedia.org/repos/product-analytics/data-pipelines/-/merge_requests/39

CX Translations and Corpora tables

KCVelaga_WMF added a subscriber: • PWaigi-WMF.May 7 2025, 10:36 AM
CodeReviewBot added a comment.May 8 2025, 3:25 PM

kcvelaga opened https://gitlab.wikimedia.org/repos/product-analytics/content-translation-airflow-jobs/-/merge_requests/4

CX Translations and Corpora job scripts

CodeReviewBot added a comment.May 8 2025, 3:25 PM

kcvelaga merged https://gitlab.wikimedia.org/repos/product-analytics/content-translation-airflow-jobs/-/merge_requests/4

CX Translations and Corpora job scripts

CodeReviewBot added a comment.May 14 2025, 9:42 AM

kcvelaga merged https://gitlab.wikimedia.org/repos/product-analytics/data-pipelines/-/merge_requests/39

CX Translations and Corpora tables

Maintenance_bot removed a project: Patch-For-Review.May 14 2025, 10:30 AM
CodeReviewBot added a project: Patch-For-Review.May 14 2025, 11:28 AM

kcvelaga opened https://gitlab.wikimedia.org/repos/product-analytics/content-translation-airflow-jobs/-/merge_requests/5

Update query commit IDs - translations and corpora

CodeReviewBot added a comment.May 14 2025, 11:29 AM

kcvelaga merged https://gitlab.wikimedia.org/repos/product-analytics/content-translation-airflow-jobs/-/merge_requests/5

Update query commit IDs - translations and corpora

Maintenance_bot removed a project: Patch-For-Review.May 14 2025, 11:30 AM
CodeReviewBot added a project: Patch-For-Review.May 15 2025, 1:58 PM

kcvelaga opened https://gitlab.wikimedia.org/repos/product-analytics/content-translation-airflow-jobs/-/merge_requests/6

Optimize CX Corpora import

CodeReviewBot added a comment.May 15 2025, 1:58 PM

kcvelaga merged https://gitlab.wikimedia.org/repos/product-analytics/content-translation-airflow-jobs/-/merge_requests/6

Optimize CX Corpora import

Maintenance_bot removed a project: Patch-For-Review.May 15 2025, 2:30 PM
KCVelaga_WMF updated the task description. (Show Details)May 16 2025, 3:11 PM
KCVelaga_WMF changed the task status from Open to In Progress.May 19 2025, 9:03 PM
KCVelaga_WMF triaged this task as Medium priority.
KCVelaga_WMF moved this task from Incoming to Priority on the LPL Analytics board.
KCVelaga_WMF moved this task from Priority to In progress on the LPL Analytics board.
KCVelaga_WMF added a project: Product-Analytics (Kanban).
CodeReviewBot added a project: Patch-For-Review.May 22 2025, 4:54 PM

kcvelaga opened https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/1348

CX Corpora monthly DAG and CX monitoring metrics daily DAG

CodeReviewBot added a comment.May 29 2025, 11:22 AM

kcvelaga merged https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/1348

CX Corpora monthly DAG and CX monitoring metrics daily DAG

Stashbot added a comment.May 29 2025, 11:26 AM

Mentioned in SAL (#wikimedia-operations) [2025-05-29T11:26:07Z] <kcvelaga@deploy1003> Started deploy [airflow-dags/analytics_product@6b26a59]: T393560

Stashbot added a comment.May 29 2025, 11:27 AM

Mentioned in SAL (#wikimedia-operations) [2025-05-29T11:27:17Z] <kcvelaga@deploy1003> Finished deploy [airflow-dags/analytics_product@6b26a59]: T393560 (duration: 01m 12s)

Maintenance_bot removed a project: Patch-For-Review.May 29 2025, 11:30 AM
KCVelaga_WMF closed this task as Resolved.Jun 2 2025, 6:05 PM
KCVelaga_WMF updated the task description. (Show Details)
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits