Page MenuHomePhabricator

Growth: validate that data is purged after 270 days
Closed, ResolvedPublic


In T237124, we implemented a wider-than-usual 270 data purge window for Growth features. The earliest data covered under that should be purged starting on August 5, 2020. On this task, we will check to make sure that's happening.

Event Timeline

LGoto triaged this task as Medium priority.Apr 10 2020, 4:36 PM
LGoto moved this task from Triage to Backlog on the Product-Analytics board.
Milimetric raised the priority of this task from Medium to High.Apr 13 2020, 3:55 PM
Milimetric moved this task from Incoming to Operational Excellence on the Analytics board.

I ran SHOW PARTITIONS event_sanitized.homepagemodule on 2020-08-07 and again today (2020-08-10). After the first run, I compared the available partitions to the date specified in T244312 (2019-11-05) and noticed that the first available partition on 2020-08-07 was 2019-11-11, indicating that data was starting to be purged.

I've also compared a diff between the two runs, which showed the correct partitions being deleted and the new ones being added.

Lastly, I checked the number of partitions available and verified that it's slightly more than 270 days. I've confirmed that this is also the case for event_sanitized.homepagevisit and event_sanitized.helppanel. So from what I can tell, this is working correctly. Handing it off to @MMiller_WMF for review.

MMiller_WMF closed this task as Resolved.Aug 17 2020, 3:02 AM

Thank you!