Page MenuHomePhabricator

Automatically monitor schema changes that would break sqoop
Open, Needs TriagePublic5 Estimated Story Points

Description

We could run the sqoop ORM generator code, once a night, for one wiki in each of our shards. If any of these break, we would know within 24 hours that we need to update sqoop queries.

This would be cheap on resources as the queries all have where 1=0.

To implement, we need to parameterize the database we use and schedule the job through Airflow. Should be relatively simple and would be decent insurance for gaps in communication with the data persistence team or cloud team.