The script deleting druid-public datasources fails silently if errors accur. Also, it doesn't handle redirects correctly (for deletes DELETE actions). Finally, it uses a to-be-deprecated deletion method for segments, and generates one action per segment (we have experienced a broker issue due to server high load after deletion of a datasource) .
This task is about refactoring the existing DruidDataDropper from therefinery-drop-druid-deep-storage-data script into the DruidUtils object.
Description
Description
Details
Details
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Refactor python util.py into smaller files | analytics/refinery | master | +1 K -1 K |
Event Timeline
Comment Actions
self.delete in utils.py should simply, in my opinion, return http exceptions if any so we can get them in the drop script and sys.exit(1)
Comment Actions
Change 502469 had a related patch set uploaded (by Joal; owner: Joal):
[analytics/refinery@master] Refactor python util.py into smaller files
Comment Actions
Change 502469 merged by Milimetric:
[analytics/refinery@master] Refactor python util.py into smaller files
Comment Actions
Ping @fdans - The patch is not listed here (don't know why), but still needs your review please: https://gerrit.wikimedia.org/r/c/analytics/refinery/+/502858