Add logic to purging scripts that requires admin action if it's about to delete a lot of data
Closed, ResolvedPublic3 Estimated Story Points
Actions

Assigned To

Authored By

	• fdans
	Dec 17 2020, 8:18 PM

Description

In any script that deletes data as part of our data retention policies, there should be some logic that evaluates if the data to be deleted looks like it shouldn't be deleted. The most obvious variable to control for would be the size in bytes of whatever will be removed. If the criteria are met, purging should be stopped and the team should be emailed.

Details

Subject	Repo	Branch	Lines +/-
analytics:refinery:job:data_purge: Improve drop-webrequest-sequence-stats	operations/puppet	production	+1 -1
Add missing changes to the deletion script	analytics/refinery	master	+57 -64
analytics:refinery:job:data_purge: Add --allowed-interval to deletion jobs	operations/puppet	production	+38 -33
Add safety limits to refinery-drop-older-than	analytics/refinery	master	+252 -126

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T267355 Traffic anomaly alarms
		Resolved		mforns	T270433 Add logic to purging scripts that requires admin action if it's about to delete a lot of data

Event Timeline

• fdans created this task.Dec 17 2020, 8:18 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 17 2020, 8:18 PM

• fdans triaged this task as High priority.Jan 4 2021, 5:26 PM

• fdans moved this task from Incoming to Security Maturity and Data Privacy on the Analytics board.

mforns claimed this task.May 5 2021, 3:54 PM

mforns added a project: Analytics-Kanban.

mforns moved this task from Next Up to In Progress on the Analytics-Kanban board.

Change 694547 had a related patch set uploaded (by Mforns; author: Mforns):

[analytics/refinery@master] Add dropped partitions and deleted directory size limits

https://gerrit.wikimedia.org/r/694547

gerritbot added a project: Patch-For-Review.May 25 2021, 4:04 PM

mforns moved this task from In Progress to In Code Review on the Analytics-Kanban board.May 25 2021, 4:07 PM

mforns added a parent task: T267355: Traffic anomaly alarms.May 27 2021, 3:05 PM

mforns added a project: Data-Engineering.Oct 22 2021, 5:49 PM

odimitrijevic added a project: Data-Engineering-Kanban.Oct 27 2021, 11:43 PM

odimitrijevic moved this task from Next Up to In Code Review on the Data-Engineering-Kanban board.Oct 27 2021, 11:45 PM

mforns moved this task from Incoming (new tickets) to Security & Governance on the Data-Engineering board.Nov 1 2021, 4:57 PM

• razzi moved this task from In Code Review to Paused on the Data-Engineering-Kanban board.Nov 30 2021, 5:13 PM

odimitrijevic removed a project: Analytics.Jan 11 2022, 11:22 PM

JArguello-WMF moved this task from Paused to In Progress on the Data-Engineering-Kanban board.May 19 2022, 5:04 PM

JArguello-WMF edited projects, added Data-Engineering-Planning; removed Data-Engineering.Jun 30 2022, 3:15 PM

JArguello-WMF edited projects, added Data-Engineering-Planning (Sprint 01); removed Data-Engineering-Planning.Jun 30 2022, 3:33 PM

JArguello-WMF moved this task from Ready to In progress on the Data-Engineering-Planning (Sprint 01) board.Jun 30 2022, 3:42 PM

• EChetty set the point value for this task to 3.Jun 30 2022, 5:17 PM

• EChetty moved this task from In progress to In code review on the Data-Engineering-Planning (Sprint 01) board.Jul 11 2022, 9:46 AM

Change 813921 had a related patch set uploaded (by Mforns; author: Mforns):

[operations/puppet@production] analytics:refinery:job:data_purge: Add --allowed-interval to deletion jobs

https://gerrit.wikimedia.org/r/813921

• EChetty moved this task from In code review to Ready to deploy on the Data-Engineering-Planning (Sprint 01) board.Jul 14 2022, 4:09 PM

• EChetty edited projects, added Data-Engineering-Planning (Sprint 02); removed Data-Engineering-Planning (Sprint 01).Jul 25 2022, 2:06 PM

• EChetty moved this task from Ready to In progress on the Data-Engineering-Planning (Sprint 02) board.Jul 25 2022, 3:29 PM

• EChetty moved this task from In progress to Ready to deploy on the Data-Engineering-Planning (Sprint 02) board.Jul 26 2022, 4:12 PM

Mentioned in SAL (#wikimedia-analytics) [2022-08-10T17:03:00Z] <ottomata> stopping puppet and drop data timers on an-launcher1002 and an-test-coord1001 to deploy drop script changes - T270433

Change 694547 merged by Ottomata:

[analytics/refinery@master] Add safety limits to refinery-drop-older-than

https://gerrit.wikimedia.org/r/694547

Mentioned in SAL (#wikimedia-operations) [2022-08-10T17:08:08Z] <otto@deploy1002> Started deploy [analytics/refinery@d4dd7e4] (hadoop-test): Add safety limits to refinery-drop-older-than - T270433 - TEST [analytics/refinery@d4dd7e4]

Change 813921 merged by Ottomata:

[operations/puppet@production] analytics:refinery:job:data_purge: Add --allowed-interval to deletion jobs

https://gerrit.wikimedia.org/r/813921

Change 822122 had a related patch set uploaded (by Mforns; author: Mforns):

[analytics/refinery@master] Add missing changes to the deletion script

https://gerrit.wikimedia.org/r/822122

Change 822122 merged by Mforns:

[analytics/refinery@master] Add missing changes to the deletion script

https://gerrit.wikimedia.org/r/822122

Maintenance_bot removed a project: Patch-For-Review.Aug 10 2022, 5:30 PM

Mentioned in SAL (#wikimedia-operations) [2022-08-10T17:30:37Z] <otto@deploy1002> Started deploy [analytics/refinery@6e47e0e] (hadoop-test): Add missing changes to the deletion script - T270433 - TEST [analytics/refinery@6e47e0e]

Mentioned in SAL (#wikimedia-operations) [2022-08-10T17:34:56Z] <otto@deploy1002> Finished deploy [analytics/refinery@6e47e0e] (hadoop-test): Add missing changes to the deletion script - T270433 - TEST [analytics/refinery@6e47e0e] (duration: 04m 19s)

Mentioned in SAL (#wikimedia-operations) [2022-08-10T17:36:40Z] <otto@deploy1002> Started deploy [analytics/refinery@6e47e0e]: Add missing changes to the deletion script - T270433 - [analytics/refinery@6e47e0e]

Mentioned in SAL (#wikimedia-operations) [2022-08-10T17:42:09Z] <otto@deploy1002> Finished deploy [analytics/refinery@6e47e0e]: Add missing changes to the deletion script - T270433 - [analytics/refinery@6e47e0e] (duration: 05m 28s)