Page MenuHomePhabricator

Update Sqoop for externallinks table changes
Closed, ResolvedPublic3 Estimated Story Points

Description

The External Links table is changing soon and we need to update our sqoop code to handle this. Some columns are going away and new columns are replacing them.

Event Timeline

JArguello-WMF set the point value for this task to 3.May 15 2023, 4:16 PM

Do you know if there is a DB with the new schema version? It would be cool to have a place to test the import.

Change 922172 had a related patch set uploaded (by Aqu; author: Aqu):

[analytics/refinery@master] Update Scoop scripts for mediawiki.externallinks schema change

https://gerrit.wikimedia.org/r/922172

Antoine_Quhen changed the task status from Open to In Progress.May 22 2023, 9:07 PM
Antoine_Quhen moved this task from In Progress to In Review on the Data Pipelines (Sprint 14) board.

Squooping test is conclusive and the patch could be merged right now.

https://phabricator.wikimedia.org/P48501

Change 922172 merged by Milimetric:

[analytics/refinery@master] Update Scoop scripts for mediawiki.externallinks schema change

https://gerrit.wikimedia.org/r/922172