|analytics/refinery : master||Remove Echo schema from EL sanitization white-list|
If it's easier, @jcrespo, you can just delete all Echo_% tables any time, we have confirmation from Roan that they don't need that data any more. I see:
If you wanted to wait until all data is 90 days old, that will happen on June 5th.
(sorry for delay, was on vacation)
It looks to me like those tables still exist and there's still data on the box that analytics-slave points to, so yeah, I think they need to be deleted. But I agree it's weird there's still data, it should've been deleted by the clean-up scripts. @mforns there's data here (for example Echo_7731316) from 2014 but it's not whitelisted, right?
The Echo schema is present in EventLogging's purging white-list, see:
Hence the purging script is keeping the following fields for all Echo schemas:
Echo clientValidated Echo event_deliveryMethod Echo event_eventSource Echo event_notificationGroup Echo event_notificationType Echo event_revisionId Echo event_sender Echo event_version
If we remove the schema from the white-list, the purging script will start removing corresponding data older than 90 days from now on, but the historical data older than 91 days as of today will still need manual purging (the purging script executes every day and only affects the 91st day, not historical data).
We could execute the purging script from the beginning of time, but I'm not sure we can restrict the tables it will process. Is it possible to limit the mysql purging script to only process the given tables, @elukey? Otherwise, it might be easier to just drop the tables.