Page MenuHomePhabricator

PII Monitoring
Open, Needs TriagePublic

Description

We should add monitoring scripts to ensure PII data is cleared from our systems, and notify us if they find something that was not anonymised properly.

Things that need to be looked for:

production DB:

  • Donations older than the expiry, and still have data in their columns or data blob
  • Memberships older than the expiry, and still have data in their columns
  • Payments older than the expiry, and still have data in their columns
  • Orphaned payments
  • Subscriptions older than the expiry, and still hold email addresses
  • Fee changes older than the expiry
  • Donation comments older than 3 years
  • Database backups older than the expiry
  • Database dumps older than 1 week
  • abandoned "marked for moderation" donations
  • abandoned "marked for moderation" memberships
  • monitor old/deactivated user accounts not cleaned up on backend.wikimedia.de

test DB:

  • same as above, some fixture/fake db entries should be kept for user testing purposes (->skipped by the monitoring)
  • monitor old/deactivated user accounts not cleaned up on test-backend.wikimedia.de

Event Timeline

I agree that we should monitor for general "out of date" cases, to monitor the functionality of our scrubbing processes.

Besides that,
I would love to keep our existing "moderation monitoring" , together with the cases you mentioned there.
I see value from it as it is a different use case than just "we need to get rid of data after expiry date",
it also helps monitoring stalled (moderation handling) processes at other teams and it helps warning us early about that.
-> the logging purpose differs a bit from the general scrubbing monitoring