Page MenuHomePhabricator
Create Task
Maniphest T166821

whitelist multimedia and upload wizard tables
Closed, ResolvedPublic5 Estimated Story Points
Actions

Description

instead of purging all the data after 90 days from the multimedia and upload wizard tables, can we simply purge the standard capsule?

I ask because with the new investment in a deeper multimedia team, it is likely these features will get attention and the historical data will be relevant:

MediaViewer (link to the others in the box on the right)
MultimediaViewerVersusPageFilePerformance
MultimediaViewerDuration
MultimediaViewerNetworkPerformance
MultimediaViewerAttribution

As far as I can see, none of these have any sensitive data outside of the standard data capsule that ships with each event. Same applies to upload wizard

UploadWizardStep (link to the others in the box on the right)
UploadWizardFlowEvent
UploadWizardErrorFlowEvent
UploadWizardExceptionFlowEvent
UploadWizardUploadFlowEvent
UploadWizardTutorialActions
UploadWizardUploadActions

Related Objects

Event Timeline

• JKatzWMF created this task.Jun 1 2017, 6:36 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 1 2017, 6:36 PM
• JKatzWMF added projects: Analytics, MediaWiki-extensions-EventLogging.Jun 1 2017, 6:37 PM
mforns edited projects, added Analytics-Kanban; removed Analytics.Jun 5 2017, 1:55 PM
mforns added a comment.Jun 5 2017, 2:13 PM

Hi @JKatzWMF!

I've looked into the schemas, and they seem non-sensitive to me, given that we purge the parsed (sanitized) userAgent field from the capsule after 90 days. This needs to be done, because some multimedia schemas contain the field 'country' and some upload schemas contain the field 'username'. Those fields combined with the sanitized userAgent can still associate location or username with a device (if the device is uncommon enough).

So, I will add the following schemas to the white-list, excluding the sanitized userAgent field, which will be purged after 90 days.
https://meta.wikimedia.org/wiki/Schema:MediaViewer
https://meta.wikimedia.org/wiki/Schema:MultimediaViewerVersusPageFilePerformance
https://meta.wikimedia.org/wiki/Schema:MultimediaViewerAttribution
https://meta.wikimedia.org/wiki/Schema:UploadWizardStep
https://meta.wikimedia.org/wiki/Schema:UploadWizardFlowEvent
https://meta.wikimedia.org/wiki/Schema:UploadWizardErrorFlowEvent
https://meta.wikimedia.org/wiki/Schema:UploadWizardExceptionFlowEvent
https://meta.wikimedia.org/wiki/Schema:UploadWizardUploadFlowEvent
https://meta.wikimedia.org/wiki/Schema:MultimediaViewerDuration
https://meta.wikimedia.org/wiki/Schema:MultimediaViewerNetworkPerformance
https://meta.wikimedia.org/wiki/Schema:UploadWizardTutorialActions
https://meta.wikimedia.org/wiki/Schema:UploadWizardUploadActions

mforns moved this task from Next Up to In Progress on the Analytics-Kanban board.Jun 5 2017, 2:20 PM
• Nuria set the point value for this task to 3.Jun 5 2017, 9:18 PM
• Nuria changed the point value for this task from 3 to 5.
• JKatzWMF added a comment.Jun 5 2017, 9:21 PM

@mforns Works for me. Thanks!

mforns moved this task from In Progress to In Code Review on the Analytics-Kanban board.Jun 6 2017, 2:58 PM

@JKatzWMF

I added the schema fields that we discussed to the white-list.
You can check it here: https://gerrit.wikimedia.org/r/#/c/298721/
The related changes are in patch set 8
Cheers!

• JKatzWMF added a comment.Jun 6 2017, 4:33 PM

Thanks, @mforns!

mforns moved this task from In Code Review to Ready to Deploy on the Analytics-Kanban board.Jun 7 2017, 2:33 PM
Milimetric triaged this task as Medium priority.Jun 22 2017, 3:09 PM
mforns added a comment.Jul 10 2017, 3:34 PM

Will move this task to done, because the editing of the white-list is finished and will be merged in a Gerrit patch belonging to another task: T156933.

mforns moved this task from Ready to Deploy to Done on the Analytics-Kanban board.Jul 10 2017, 3:35 PM
• Nuria closed this task as Resolved.Jul 11 2017, 6:41 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL