Maniphest T199898

EventLogging sanitization
Closed, ResolvedPublic0 Estimated Story Points
Actions

Assigned To

Authored By

	mforns
	Jul 18 2018, 11:11 AM

Tags

Referenced Files

None

Subscribers

Description

This is a parent task for all work related to EventLogging sanitization that remains to consider this project fully done.

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		mforns	T199898 EventLogging sanitization
		Declined		mforns	T178174 [EL sanitization] Remove AppInstallIId from EventLogging purging white-list
		Resolved		mforns	T199836 [EL sanitization] Write and productionize script to drop partitions older than 90 days in events database
		Resolved		mforns	T198426 [EL sanitization] Add ability to salt and hash to eventlogging sanitization in Hive
		Declined		None	T199230 [EL sanitization] Make WhitelistSanitization support arrays of structs, maps or other arrays
		Resolved		mforns	T199899 [EL Sanitization] Set up salt creation and rotation
		Resolved		mforns	T199900 [EL sanitization] Store the old salt for 2 extra weeks
		Resolved		mforns	T199902 [EL sanitization] Retroactively sanitize (including hash and salt appInstallId fields) data in the events database
		Resolved		mforns	T202429 [EL sanitization] Make cron send alert emails if job fails before calling refine
		Resolved		mforns	T209503 [EventLogging Sanitization] Enable older-than-90-day purging of unsanitized EL database (event) in Hive

Event Timeline

mforns created this task.Jul 18 2018, 11:11 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 18 2018, 11:11 AM

mforns claimed this task.Jul 18 2018, 11:11 AM

mforns set the point value for this task to 0.

mforns moved this task from Next Up to Parent Tasks on the Analytics-Kanban board.

mforns added a subtask: T178174: [EL sanitization] Remove AppInstallIId from EventLogging purging white-list.

mforns added subtasks: T199836: [EL sanitization] Write and productionize script to drop partitions older than 90 days in events database, T198426: [EL sanitization] Add ability to salt and hash to eventlogging sanitization in Hive, T199230: [EL sanitization] Make WhitelistSanitization support arrays of structs, maps or other arrays.Jul 18 2018, 11:19 AM

mforns added a subtask: T199899: [EL Sanitization] Set up salt creation and rotation.Jul 18 2018, 11:23 AM

mforns added a subtask: T199900: [EL sanitization] Store the old salt for 2 extra weeks.Jul 18 2018, 11:26 AM

Milimetric triaged this task as Medium priority.Jul 19 2018, 3:05 PM

Milimetric moved this task from Incoming to Data Quality on the Analytics board.Jul 19 2018, 3:40 PM

Wesley8086 moved this task from Parent Tasks to Next Up on the Analytics-Kanban board.Aug 15 2018, 10:02 AM

Wesley8086 moved this task from Next Up to Parent Tasks on the Analytics-Kanban board.

Milimetric mentioned this in T178174: [EL sanitization] Remove AppInstallIId from EventLogging purging white-list.Aug 20 2018, 4:20 PM

Milimetric closed subtask T178174: [EL sanitization] Remove AppInstallIId from EventLogging purging white-list as Declined.

• Nuria closed subtask T199899: [EL Sanitization] Set up salt creation and rotation as Resolved.Aug 22 2018, 10:46 PM

• Nuria closed subtask T198426: [EL sanitization] Add ability to salt and hash to eventlogging sanitization in Hive as Resolved.Aug 30 2018, 9:55 PM

dr0ptp4kt mentioned this in T205569: Define cross-schema event stitching approach.Oct 12 2018, 5:03 PM

• Nuria closed subtask T199900: [EL sanitization] Store the old salt for 2 extra weeks as Resolved.Oct 19 2018, 2:23 AM

• Nuria closed subtask T199902: [EL sanitization] Retroactively sanitize (including hash and salt appInstallId fields) data in the events database as Resolved.Oct 19 2018, 2:48 AM

Milimetric removed a project: Analytics.Oct 19 2018, 4:15 PM

• jlinehan subscribed.Dec 19 2018, 4:23 PM

• Nuria closed subtask T202429: [EL sanitization] Make cron send alert emails if job fails before calling refine as Resolved.Feb 14 2019, 5:10 AM

• Nuria closed subtask T199836: [EL sanitization] Write and productionize script to drop partitions older than 90 days in events database as Resolved.Feb 14 2019, 5:19 AM

• Nuria closed subtask T209503: [EventLogging Sanitization] Enable older-than-90-day purging of unsanitized EL database (event) in Hive as Resolved.Mar 11 2019, 6:05 PM

The only subtask, supporting arrays and maps, is separate from this main effort, declaring victory. Well done!

JArguello-WMF closed subtask T199230: [EL sanitization] Make WhitelistSanitization support arrays of structs, maps or other arrays as Declined.Jul 4 2022, 6:14 PM

Restricted Application added a project: Data-Engineering. · View Herald TranscriptJul 4 2022, 6:15 PM