Page MenuHomePhabricator
Create Task
Maniphest T199836

[EL sanitization] Write and productionize script to drop partitions older than 90 days in events database
Closed, ResolvedPublic8 Estimated Story Points
Actions

Description

Drop older than 90 days partitions in events database and create documentation that points to events-sanitized database with examples.

Details

ProjectBranchLines +/-Subject
analytics/refinerymaster+540 -0Add bin/refinery-drop-older-than
Customize query in gerrit

Related Objects
Search...

Event Timeline

Nuria created this task.Jul 17 2018, 8:38 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 17 2018, 8:38 PM
Nuria renamed this task from Hadoop Sanitization. Drop older than 90 days partitions in events database to Hadoop Sanitization. Drop partitions older than 90 days in events database .Jul 17 2018, 8:40 PM
mforns renamed this task from Hadoop Sanitization. Drop partitions older than 90 days in events database to [EL sanitization] Write and productionize script to drop partitions older than 90 days in events database.Jul 18 2018, 11:02 AM
mforns added a parent task: T199898: EventLogging sanitization.Jul 18 2018, 11:19 AM
Nuria assigned this task to mforns.Aug 22 2018, 10:47 PM
mforns moved this task from Next Up to In Progress on the Analytics-Kanban board.Oct 17 2018, 3:16 PM
Milimetric added a project: Analytics.Oct 19 2018, 4:17 PM
Milimetric triaged this task as High priority.Oct 19 2018, 4:25 PM
Milimetric moved this task from Incoming to Data Quality on the Analytics board.
gerritbot added a subscriber: gerritbot.Nov 2 2018, 3:58 PM

Change 471279 had a related patch set uploaded (by Mforns; owner: Mforns):
[analytics/refinery@master] Add bin/refinery-drop-older-than

https://gerrit.wikimedia.org/r/471279

gerritbot added a project: Patch-For-Review.Nov 2 2018, 3:58 PM
mforns moved this task from In Progress to In Code Review on the Analytics-Kanban board.Nov 13 2018, 2:18 PM
mforns mentioned this in T209503: [EventLogging Sanitization] Enable older-than-90-day purging of unsanitized EL database (event) in Hive.Nov 14 2018, 4:34 PM
gerritbot added a comment.Nov 29 2018, 11:26 AM

Change 471279 merged by Mforns:
[analytics/refinery@master] Add bin/refinery-drop-older-than

https://gerrit.wikimedia.org/r/471279

mforns moved this task from In Code Review to Done on the Analytics-Kanban board.Dec 11 2018, 4:06 PM
Nuria closed this task as Resolved.Feb 14 2019, 5:19 AM
Nuria set the point value for this task to 8.
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL