Page MenuHomePhabricator

Sqoop will fail on 2022-09-01 unless we fix templatelinks query
Open, HighPublic3 Estimated Story PointsBUG REPORT

Description

Running the sqoop-whole-mediawiki process will fail for the next few months as the templatelinks table is being changed. The schema is different, so we have to alter the hive table and python logic/sqoop queries so the next run doesn't fail.

This has a hard deadline of 2022-08-30

Event Timeline

Sounds like a sub task of T304979 or at least related

Change 821312 had a related patch set uploaded (by Milimetric; author: Milimetric):

[analytics/refinery@master] Adapt to templatelinks schema changes

https://gerrit.wikimedia.org/r/821312

EChetty triaged this task as High priority.Tue, Aug 16, 2:44 PM
EChetty set the point value for this task to 3.Tue, Aug 16, 2:51 PM
EChetty removed the point value for this task.
EChetty set the point value for this task to 3.
EChetty moved this task from To be discussed to Sprint 00 on the Data Pipelines board.
EChetty edited projects, added Data Pipelines (Sprint 00); removed Data Pipelines.