Add mediawiki_skin_diff to the allowlist
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	MNeisler
	Jul 23 2021, 3:17 PM

Description

We need to add non-PII fields from the new mediawiki_pref_diff schema (created in T261842) to the allowlist so that the data won't get purged after 90 days.

Later we decided to configure a new stream specifically for skin-related preference change events: mediawiki_skin_diff (T289622)

With the new instrumentation, here is our proposal of event sanitization.

Proposal

What NOT to keep

user_agent_map
http
meta:
- request_id
- id
- domain
- dt

What to hash

user_hash

What to keep

initial_state
final_state
bucketed_user_edit_count
normalized_host

Done

https://gerrit.wikimedia.org/r/c/analytics/refinery/+/737471

Details

	Project	Branch	Lines +/-	Subject
	analytics/refinery	master	+7 -0	Add skin_diff schema to sanitize allowlist
	analytics/refinery	master	+9 -0	Add the mediawiki_pref_diff event platform stream to the allowlist

Customize query in gerrit

Related Objects

Mentioned In: T289622: Create new stream to log events from VectorPrefDiffInstrumentation
T281511: Add content_translation_event data stream to the sanitization allowlist
Mentioned Here: T289622: Create new stream to log events from VectorPrefDiffInstrumentation
T261842: Create schema to track users opting in/out of desktop improvements

Event Timeline

MNeisler created this task.Jul 23 2021, 3:17 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 23 2021, 3:17 PM

MNeisler triaged this task as High priority.Jul 23 2021, 3:18 PM

MNeisler updated the task description. (Show Details)Jul 23 2021, 3:22 PM

nshahquinn-wmf mentioned this in T281511: Add content_translation_event data stream to the sanitization allowlist.Aug 6 2021, 5:39 PM

MNeisler moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.Aug 13 2021, 7:53 PM

Change 713644 had a related patch set uploaded (by MNeisler; author: MNeisler):

[analytics/refinery@master] Add the mediawiki_pref_diff event platform stream to the allowlist

https://gerrit.wikimedia.org/r/713644

gerritbot added a project: Patch-For-Review.Aug 18 2021, 3:23 PM

MNeisler added a project: Analytics.Aug 18 2021, 3:29 PM

MNeisler moved this task from Doing to Needs Review on the Product-Analytics (Kanban) board.

odimitrijevic moved this task from Incoming to Event Platform on the Analytics board.Aug 20 2021, 8:28 PM

MNeisler mentioned this in T289622: Create new stream to log events from VectorPrefDiffInstrumentation.Aug 24 2021, 8:25 PM

Change 713644 abandoned by MNeisler:

[analytics/refinery@master] Add the mediawiki_pref_diff event platform stream to the allowlist

Reason:

Per discussions with Mikhail, we decided it would be worthwhile to configure a new stream specifically for skin-related preference change events sent by the VectorPrefDiffInstrumentation. This will help ensure that changes to other preferences are kept in a separate stream and different retention policies can be applied. I'm abandoning this change and will submit a new patch once the new stream is configured. Thanks all!

https://gerrit.wikimedia.org/r/713644