Page MenuHomePhabricator

Swift for differential privacy data publication
Open, Needs TriagePublic

Description

I am currently leading a cross-departmental team working on implementing a general-purpose differential privacy engine for safer data releases. We are installing the necessary software, developing personal expertise, creating policy/process around differential privacy, etc. — all with the goal of increasing the safety and granularity of sensitive data that we release. Right now, all of the technical development is taking place within the analytics cluster.

One important part of figuring out the differential privacy process is getting finished tables from the private analytics cluster to an accessible place where the public can download them. In some cases, we may be publishing tables with hundreds of thousands of entries on a daily basis. We were hoping to use swift as a data publication service, and wanted to put that request on your radar. Let me know if there are any ways I can be helpful as we discuss/figure out this process!

Event Timeline

@Htriedman: I know you're talking to @EChetty about this, we're triaging it to this column which is like a task "incubator". Once this is fully formed and we know what the pipeline looks like, we can help you expand this into the necessary tasks. When you're done, you can move this back to incoming to effectively ping us.

@Milimetric Thanks for the pointers on this process! I also just talked to @gmodena and think that we're starting to come to a good set of solutions for how we might put together the disparate pieces of this project. I'll be sure to keep you all in the loop.