Page MenuHomePhabricator

Create new stream to log events from VectorPrefDiffInstrumentation
Closed, ResolvedPublic3 Estimated Story Points

Description

Request: The VectorPrefDiffInstrumention created in T261842 currently sends all events to a stream called mediawiki.pref_diff. We'd like to create a new stream that is more specifically named (such as, mediawiki.skin_switch) to send these events to instead.

Context: The current stream name mediawiki.pref_diff seems like a generic, catch-all stream for all preference change events; however, its purpose is only to log user-initiated skin changes from the VectorPrefDiff Instrument. It's possible that a different instrument could use the same stream and log changes to other unrelated user preferences such as email. This is problematic as other preferences should be logged in separate streams so separate retention policies can be applied. For example, we'd like to retain skin switch data past 90 days but may not want to retain other user-sensitive properties past 90 days. As a result, we recommend changing the stream name for these events to something more specific to avoid confusion.

Suggested Steps:

  1. Create new stream entry in mediawiki config file. Proposed new name: mediawiki.skin_switch or mediawiki.skin_diff
  2. Change stream name in VectorPrefDiffInstrumentation
  3. Update the Desktop Improvements Opt-Out Chart in Superset Dashboard
  4. Add to the allowlist (Will be done in T287255)

Developer Notes:

  • Per T289622#7354305, this doesn't require any changes to the schema. We should only change the STREAM_NAME constant in VectorPrefDiffInstrumentation as well as the stream name in mediawiki-config.

Signoff criteria

  • Create spike that explores how instrumentation relates to proposed changes to skin version setting

Event Timeline

MNeisler renamed this task from Create new stream to log events from VectorPrefDiff Instrument to Create new stream to log events from VectorPrefDiffInstrumentation.Aug 24 2021, 8:18 PM
MNeisler updated the task description. (Show Details)
MNeisler moved this task from Triage to Tracking on the Product-Analytics board.

Change 717622 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[schemas/event/secondary@master] POC: add new stream for VectorPrefDiffInstrumentation

https://gerrit.wikimedia.org/r/717622

Change 717623 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[mediawiki/extensions/WikimediaEvents@master] POC: update stream for VectorPrefDiffInstrumentation

https://gerrit.wikimedia.org/r/717623

cjming removed cjming as the assignee of this task.Sep 3 2021, 9:46 PM
cjming updated the task description. (Show Details)
cjming subscribed.

@MNeisler @jwang Is the ask of this ticket to literally only change the stream name and not anything else (e.g. the schema and the usage of "pref_diff" in it is fine, the VectorPrefDiffInstrumentation class name doesn't need to change)?

From my understanding of the ticket, changing the stream name will hopefully discourage others from using the same stream name and will also enable a different data retention policy, is that correct?

@nray, yes, your understanding is correct.

hi @jwang - I'm still fairly new to the organization so apologies if this question has an obvious answer - how does one confirm that the new stream name is collecting the data you're requesting?

We can swap out the stream name fairly easily for this instrumentation class but I'm not clear on how to verify that it's working properly.

Hi @cjming I am not an expert in this area. I guess if the instrumentation works, we should see data in new table event.mediawiki_skin_diff. And the recent data there should be same as the data in event.mediawiki_pref_diff. I didn't see the table event.mediawiki_skin_diff yet. I can help with QA when the new schema is available.

nray updated the task description. (Show Details)

Change 725161 had a related patch set uploaded (by Nray; author: Nray):

[operations/mediawiki-config@master] Add new 'mediawiki.skin_diff' event logging stream

https://gerrit.wikimedia.org/r/725161

Change 725162 had a related patch set uploaded (by Nray; author: Nray):

[mediawiki/extensions/WikimediaEvents@master] Change VectorPrefDiffInstrumentation stream name to `mediawiki.skin_diff`

https://gerrit.wikimedia.org/r/725162

The above two patch are ready for review and can be backported when ready. I am OOO Oct 1 - 4, but will plan on backporting next week if it hasn't been done already

@phuedx Will you have time to review the above two patches this week?

Thanks for the ping, @nray. I'm sorry I didn't look at this sooner. I've +2'd https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/725162 and +1'd https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/725161. I'm happy to help deploy the latter or you can schedule it for deployment at a time convenient to you.

Thank you @phuedx! I'm happy to schedule a backport for it, I had just one more question for you regarding a reference I found in the schema example:

https://gerrit.wikimedia.org/g/schemas/event/secondary/+/74059819afe4dfc7022fd2836edf949af8a63f8e/jsonschema/analytics/pref_diff/current.yaml#51

Is it necessary to update the stream name in that example block as well to mediawiki.skin_diff or is it truly just an example?

https://gerrit.wikimedia.org/g/schemas/event/secondary/+/74059819afe4dfc7022fd2836edf949af8a63f8e/jsonschema/analytics/pref_diff/current.yaml#51

Is it necessary to update the stream name in that example block as well to mediawiki.skin_diff or is it truly just an example?

It is an example and, for now, it could stay that way without much confusion since both streams are still configured.

Change 730732 had a related patch set uploaded (by Nray; author: Nray):

[mediawiki/extensions/WikimediaEvents@wmf/1.38.0-wmf.3] Change VectorPrefDiffInstrumentation stream name to `mediawiki.skin_diff`

https://gerrit.wikimedia.org/r/730732

Change 730733 had a related patch set uploaded (by Nray; author: Nray):

[mediawiki/extensions/WikimediaEvents@wmf/1.38.0-wmf.4] Change VectorPrefDiffInstrumentation stream name to `mediawiki.skin_diff`

https://gerrit.wikimedia.org/r/730733

Change 730733 abandoned by Nray:

[mediawiki/extensions/WikimediaEvents@wmf/1.38.0-wmf.4] Change VectorPrefDiffInstrumentation stream name to `mediawiki.skin_diff`

Reason:

On second thought, I'll let it ride the train

https://gerrit.wikimedia.org/r/730733

Change 730732 abandoned by Nray:

[mediawiki/extensions/WikimediaEvents@wmf/1.38.0-wmf.3] Change VectorPrefDiffInstrumentation stream name to `mediawiki.skin_diff`

Reason:

On second thought, I'll let it ride the train

https://gerrit.wikimedia.org/r/730732

Thank you @phuedx, I will backport the config today (in the next hour)

nray added a subscriber: phuedx.

Moving this to blocked as it is blocked on me backporting

Change 725161 merged by jenkins-bot:

[operations/mediawiki-config@master] Add new 'mediawiki.skin_diff' event logging stream

https://gerrit.wikimedia.org/r/725161

Mentioned in SAL (#wikimedia-operations) [2021-10-14T18:07:17Z] <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 41baa8c41d64510986f009b9be2d70dad0915f8c: Add new mediawiki.skin_diff event logging stream (T289622) (duration: 01m 05s)

Change 725162 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] Change VectorPrefDiffInstrumentation stream name to `mediawiki.skin_diff`

https://gerrit.wikimedia.org/r/725162

Change 730733 restored by Nray:

[mediawiki/extensions/WikimediaEvents@wmf/1.38.0-wmf.4] Change VectorPrefDiffInstrumentation stream name to `mediawiki.skin_diff`

https://gerrit.wikimedia.org/r/730733

Change 730733 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@wmf/1.38.0-wmf.4] Change VectorPrefDiffInstrumentation stream name to `mediawiki.skin_diff`

https://gerrit.wikimedia.org/r/730733

Mentioned in SAL (#wikimedia-operations) [2021-10-14T23:34:05Z] <cjming@deploy1002> Synchronized php-1.38.0-wmf.4/extensions/WikimediaEvents/includes/VectorPrefDiffInstrumentation.php: Backport: [[gerrit:730733|Change VectorPrefDiffInstrumentation stream name to mediawiki.skin_diff (T289622)]] (duration: 00m 56s)

@jwang The stream name has been updated to mediawiki.skin_diff and deployed everywhere . Can you please check to make sure everything is okay? Also, I'm assuming steps 3 and 4 of the suggested steps are to be done by you (I actually don't have access to see that chart)?

nray reassigned this task from jwang to Edtadros.
nray reassigned this task from Edtadros to jwang.
nray added a subscriber: Edtadros.

@nray thank you. You are correct. I will work on step 3 and 4.

Change 717623 abandoned by Clare Ming:

[mediawiki/extensions/WikimediaEvents@master] POC: update stream for VectorPrefDiffInstrumentation

Reason:

https://gerrit.wikimedia.org/r/717623

Moving this to signoff as I believe @jwang has already checked that the data is flowing properly via the new stream

Change 717622 abandoned by Clare Ming:

[schemas/event/secondary@master] POC: add new stream for VectorPrefDiffInstrumentation

Reason:

https://gerrit.wikimedia.org/r/717622