| filename: | * |
| what the file is for: | fundraising |
| what is the sensitive data: | raw logs |
| is it being updated: | no |
| recommendations: | Making sure fundraising is no longer using them, and then deleting them. |
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | None | T83531 Implement Data Retention Guidelines | |||
| Restricted Task | |||||
| Restricted Task | |||||
| Resolved | akosiaris | T92336 Delete gadolinium:/a/log/fundraising/ | |||
| Restricted Task | |||||
| Restricted Task | |||||
| Restricted Task | |||||
| Restricted Task | |||||
| Restricted Task | |||||
| Restricted Task | |||||
| Restricted Task | |||||
| Restricted Task | |||||
| Restricted Task | |||||
| Resolved | Jgreen | T110592 package udp-filter for Trusty, for use on fundraising banner_logger | |||
| Resolved | Jgreen | T110740 build libcidr package for Trusty | |||
| Resolved | Jgreen | T110739 build libanon package for trusty | |||
| Resolved | Jgreen | T112139 build librdkafka for Trusty | |||
| Resolved | Jgreen | T110591 reformulate kafkatee package to work with Trusty | |||
| Restricted Task | |||||
| Resolved | awight | T97676 Verify kafkatee use for fundraising logs on erbium | |||
| Resolved | coren | T97860 Add andyrussg to udp2log-users group to allow him to verify kafkatee generated fundraising log files on erbium | |||
| Resolved | awight | T116800 Impression log parsers should get sample rate from filenames |
Event Timeline
Comment Actions
the only thing at /a/log/fundraising is the nfs mount from the netapp, which is still in use for log collection. the banner log collection pipeline has to be fully migrated before we delete/umount this.
Comment Actions
@Jgreen these are a really naive questions: does it still need to be mounted here? can we unmount it?
Comment Actions
I just sent Jeff an email about this. I'm pretty sure these can be deleted, because fundraising logs are collected and rotated on erbium right now. I will wait for him to confirm.
Comment Actions
Ah, I forgot that we'd moved collection to erbium. It's ok to unmount the netapp and delete the dirs here.
Comment Actions
According to https://phabricator.wikimedia.org/rOPUP6ce3f053b74cd1fe2525f4550a3f81753daa8277, gadolinium as well as erbium are now in a spare status. Unfortunately we haven't wiped these hosts yet and hence neither the umount has taken place, neither the deletion of the files. I 'll create tasks to wipe the 2 systems which will effectively do part of the above. That was the last blocker to fully remove the netapps from our infrastruture and reclaim the rack space they use.
Comment Actions
T123029 tracked the above. Now erbium, gadolinium are no longer active. The unmount has taken place and the netapp is getting ready for decomissioning. The data will get deleted in about a week or so (just a prudent waiting time).