Page MenuHomePhabricator

Create schema to track users opting in/out of desktop improvements
Open, HighPublic5 Estimated Story Points

Description

Background

Due to some issues with the preference update schema, we will need to create a schema that will only track the opt-ins and opt-outs of desktop improvements. This task tracks the creation of that schema.

Target

We are hoping to see less than 40% of logged-in users opting out of the new experience

Acceptance criteria

We will need the following

  • User ID
  • Initial state of skin version
  • Final state of skin version
  • User edit bucket (if applicable)

Developer notes

This schema already exists, but doesn't quite do what we want it to do, so implementation would essentially be copying the existing preference logging in WikimediaEvents, via the same hooks, but specifically looking for the skin version option.

For purpose of estimation, it would probably make sense to update PrefUpdateInstrumentation in WikimediaEvents with some desktop refresh special handling, logged to another schema.

Using onPreferencesFormPreSave to distinguish between the switched skin option.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Is this not already done given PrefUpdate exists and is working today?

What additional high-level product questions, if any, are needed to be answered?

@Krinkle - see my comment in T261842#6759083. We would like to be able to determine the number of users that are opting in and out of desktop improvements on their own. Right now, PrefUpdate cannot distinguish between these users and the ones that are using the new version of vector by default (because we've changed the configuration on that wiki).

Change 668529 had a related patch set uploaded (by Nray; owner: Nray):
[schemas/event/secondary@master] Add new analytics/skin_change schema

https://gerrit.wikimedia.org/r/668529

Change 668530 had a related patch set uploaded (by Nray; owner: Nray):
[mediawiki/extensions/WikimediaEvents@master] WIP: Add onPreferencesFormPreSave hook to track Vector skin version toggle change

https://gerrit.wikimedia.org/r/668530

@ovasileva could you expand on "Due to some issues with the preference update schema" - it's hard to analyze this without fully understanding/defining what's not working for us and what will work differently.

I believe that, since the changes from last year, the preferences schema currently cannot tell between a default that is set for a specific feature and a change in preference - as in, an opt-in by default and an opt-in by choice look the same within the schema. Our schema would distinguish between these two states.

Chiming in here as a co-maintainer of the PrefUpdate schema and WikimediaEvents. I agree that PrefUpdate is broken (see, e.g., T218835) and probably better avoided, but I don't think it's broken in the particular way that you're describing. A change in wiki default settings should not result in PrefUpdate events being created, and doesn't, as far as I know.

@Mholloway From my understanding, I think at least part of the problem is that the default setting for the property we are interested in, VectorSkinVersion, is dependent on context and is not static like most other user properties. That is, its default value is dependent on whether the user is a newly created user or an existing user and the values of two related config variables in Vector - VectorDefaultSkinVersionForExistingAccounts and VectorDefaultSkinVersionForNewAccounts [1].

Because the PrefUpdateInstrumentation is currently and understandably unaware of these anomalies, it is always setting the isDefault property to false [2] which is incorrect for our use case. I think another problem may be occurring when a user switches from legacy Vector to another skin (e.g. Monobook). In that case, the legacy Vector checkbox becomes disabled and because it is technically not checked anymore, PrefUpdateInstrumentation emits an event that makes it look like the user opted-in to new Vector (setting VectorSkinVersion to 2) when in fact they changed to another skin. Edit: Looks like the onPreferencesFormPreSave hook in Vector prevents the preceding situation from happening

Please also see @phuedx's two comments on another ticket for further insight [3][4].

[1] https://github.com/wikimedia/Vector/blob/c9c00e71b0f1e2b199a97e72b9dde189c5a5b720/skin.json#L243-L250
[2] https://phabricator.wikimedia.org/T258971
[3] https://phabricator.wikimedia.org/T258971#6348418
[4] https://phabricator.wikimedia.org/T258971#6402912

@MNeisler I'm still fuzzy on what is desired with this ticket and was hoping you could shed some light on the following questions I have:

  1. Can you please clarify how you were using the isDefault field in T258971 and why it is important for the analysis? I understand it was/is erroneous (see above), but it's still not clear to me why it was/is important for our use case (and sorry for possibly being dense!). If it helps at all, it looks to me like the PrefUpdateInstrumentation will only send events if the user initiated the preference change [1] so it's not clear to me where the inability to distinguish users who have opted-in/opted-out by choice comes from [2]
  1. Assuming we go with schema proposed in this ticket, can you please define the properties "Initial state of skin version" and "Final state of skin version" and give examples of the values you are looking for with those properties? For example, does initial mean the default preference (which can vary depending on whether the user is a newly registered user or an existing user) or does initial mean the state of the checkbox before it was toggled (and if that is the case why is that important?)
  1. If a user has the "Use Legacy Vector" checkbox unchecked, but then switches to another skin (e.g. Monobook), should we track this? If so, what should the values of "Initial state of skin version" and "Final state of skin version" be? Similarly, what should happen if a user has the "Use Legacy Vector" checkbox checked, but then switches to another skin?
  1. If a user switches to the "Vector" skin from another skin (e.g. Monobook), should we track this? If so, what should the values of "Initial state of skin version" and "Final state of skin version" be?

Thank you for your help! 😃

/cc @ovasileva

[1] https://github.com/wikimedia/mediawiki-extensions-WikimediaEvents/blob/master/includes/PrefUpdateInstrumentation.php#L119-L121
[2] https://phabricator.wikimedia.org/T261842#6860118

Thanks for the explanation and pointers, @nray. That clears things up.

I should mention that I suspect you'll see a much lower number of events for this new schema than with PrefUpdate, because (as I describe on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MobileFrontend/+/665236) I believe that massive numbers of spurious PrefUpdate events, including for VectorSkinVersion, are currently being issued by the PrefUpdate instrumentation. The number created by this new instrumentation should much better reflect reality.

@MNeisler I'm still fuzzy on what is desired with this ticket and was hoping you could shed some light on the following questions I have:

@nray (cc @ovasileva ) I posted some thoughts and recommendations below. Sorry for the long post but hopefully this helps! Let me know if you have any questions or other suggestions.

  1. Can you please clarify how you were using the isDefault field in T258971 and why it is important for the analysis? I understand it was/is erroneous (see above), but it's still not clear to me why it was/is important for our use case (and sorry for possibly being dense!). If it helps at all, it looks to me like the PrefUpdateInstrumentation will only send events if the user initiated the preference change [1] so it's not clear to me where the inability to distinguish users who have opted-in/opted-out by choice comes from [2]

You're right that the isDefault field is technically not required to determine the number of user-initiated preference changes to a certain skin version. For example, we could just count the number of events or distinct users where the final state of the skin version = 'legacy' to determine how many people manually changed their preference to the old vector skin version.

However, the isDefault field is useful for confirming which skin value corresponds to the default state for that user, which we can't assume based on the user's skin selection since a user can opt in and out of different skins multiple times. For example, if we have an event where the final skin state is logged as 'legacy', it’s difficult to know without the isDefault field if that user was (1) presented the latest skin vector as default or (2) presented the legacy skin as default, switched to the latest skin, and then decided to switch back. We also would not be able to confirm this by looking at their first logged change since changes that occurred prior to 90 days are sanitized.

I could use my understanding of the default experience deployed to each user type when reviewing the data but it would be preferable to add this context to the instrumentation itself to simplify the analysis and help catch any data anomalies.

  1. Assuming we go with schema proposed in this ticket, can you please define the properties "Initial state of skin version" and "Final state of skin version" and give examples of the values you are looking for with those properties? For example, does initial mean the default preference (which can vary depending on whether the user is a newly registered user or an existing user) or does initial mean the state of the checkbox before it was toggled (and if that is the case why is that important?)

I think it depends on if we decide to use this schema to record user preference changes only for the vector skin versions or expand to include other types of skins. If we only use to record changes to the new vector skin, there’s no need to track an initial state (state of checkbox before it was toggled) since there are only two options and I think we could use something similar to what I proposed below:

Vector-Version Only Schema
userid
defaultVersion: Default skin presented to user
selectedVersion: State of checkbox after it was toggled

If we include other skin types (i.e. monobook), then I'd recommend the following to clarify the state of the desktop skin before the change.

All Desktop Skin Types Schema
userid
defaultState: Default skin presented to user
Initial state of skin: State of desktop skin before user changed preferences
Final state of skin: State of desktop skin after user changed preferences

  1. If a user has the "Use Legacy Vector" checkbox unchecked, but then switches to another skin (e.g. Monobook), should we track this? If so, what should the values of "Initial state of skin version" and "Final state of skin version" be? Similarly, what should happen if a user has the "Use Legacy Vector" checkbox checked, but then switches to another skin?

I think we should track this type of event. Even though they didn't switch to one of the vector versions, it's still technically a type of opt-out and would be good to account for in the data. I think using the "All Desktop Skin Types Schema" would allow us to see the frequency this type of event happens and filter out or include in the desktop opt-in/opt-out rate as needed.

Using the “All Desktop Skin Types” schema option above, this would be recorded as:

defaultState: [Will vary based on user]
Initial state of skin: Latest Vector
Final state of skin: Monobook

Similarly, what should happen if a user has the "Use Legacy Vector" checkbox checked, but then switches to another skin?

defaultState: [Will vary based on user]
Initial state of skin: Legacy Vector
Final state of skin: Monobook [or other skin type]

  1. If a user switches to the "Vector" skin from another skin (e.g. Monobook), should we track this? If so, what should the values of "Initial state of skin version" and "Final state of skin version" be?

Yes, I think we should track for the same reasons I mentioned in response to question #3 above. We can always filter these instances out if we don't want to include in our opt-out rate but useful to have the data.

defaultState: [Will vary based on user]
Initial state of skin: Monobook
Final state of skin: Legacy or Latest [will vary based on user]

Thank you for your help! 😃

/cc @ovasileva

[1] https://github.com/wikimedia/mediawiki-extensions-WikimediaEvents/blob/master/includes/PrefUpdateInstrumentation.php#L119-L121
[2] https://phabricator.wikimedia.org/T261842#6860118

@MNeisler Thank you for clarifying! That helped clear up a lot for me.

I think the "All Desktop Skin Types Schema" is doable. If I'm understanding correctly, it sounds like the default version of the skin (defined by the logged-in state of the skin before the user has made any user-initiated skin change) is desired info to send with each event (please correct if needed). FWIW, we don't currently persist or have ever persisted that default state in the database so to start sending that info with each event we could either:

A) Rely on the current state of our config variables, wgVectorDefaultSkinVersionForExistingAccounts and wgVectorDefaultSkinVersionForNewAccounts, to infer the default state for the user whenever the user opts-in or opts-out. In theory, those two config variables can be set to different values for any given wiki, but I don't believe this has ever been the case in practice (@ovasileva are there future plans to do this?). We could send the current state of one those variables (e.g. wgVectorDefaultSkinVersionForExistingAccounts or we could merge those config variables into one e.g. wgVectorDefaultSkinVersionForLoggedInUsers and send that) in each event whenever a user explicitly opts-in or opts-out. One of the apparent disadvantages to this approach is that the "default" could change if the config variable changed (i.e. events for the same user id could have different "default" states depending on what the value of the config was at the time that the event was sent and may not reflect what the user actually saw as the default skin before they changed skins though the preferences). This might happen if a wiki gets promoted to be a pilot wiki or if we changed the value for any other reason and a user opt-in/opts-out multiple times, for example. Therefore, the "defaultState" field would be more accurately named "configState" with this approach.

B) Start saving that info in the database. Existing users who have never explicitly opted-in or opted-out could have a record added to the database that identifies that default when they first explicitly opt-in or opt-out. Newly registered users could also have a record added to the database that identifies the default when they registered. The default state for users who have already explicitly opted-in or opted-out would be harder to determine since we never persisted this info although maybe we could fallback to one of the config variables.

A is easier than B, but I think both approaches are doable. Would A and its described drawbacks be acceptable/sufficient though?

Also ccing @phuedx in case he has any other ideas here 😃

Moving this to blocked per my question above (https://phabricator.wikimedia.org/T261842#6899225) regarding whether A is sufficient in which we would always send the current state of the config for each event (the easiest option) or whether we should pursue B

nray removed nray as the assignee of this task.Mar 11 2021, 5:28 PM
nray added a subscriber: nray.

Sorry, forgot to assign to @MNeisler yesterday

@MNeisler @ovasileva and I had a meeting this week in which we decided to drop the proposed "defaultState" field, in favor of only sending the initial and final state of the skin in each event. An example of such event would look like (please note the before and after fields that log the skin state and vector2 denoting a switch from latest vector to minerva):

event: {
  "meta": {
    "domain": "localhost",
    "stream": "mediawiki.pref_diff",
    "id": "6bc510f4-afd5-4201-8128-81ad69a6acd9",
    "dt": "2021-03-18T17:45:42.044Z",
    "request_id": "c24ab6b0-8811-11eb-a567-ea5c40032ec1"
  },
  "http": {
    "request_headers": {
      "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.82 Safari/537.36"
    }
  },
  "$schema": "/analytics/pref_diff/1.0.0",
  "user_id": 1,
  "before": "vector2",
  "after": "minerva",
  "bucketed_user_edit_count": "100-999 edits",
  "dt": "2021-03-18T17:45:42Z"
}

A user switching from legacy to latest vector would look like:

event: {
  "meta": {
    "domain": "localhost",
    "stream": "mediawiki.pref_diff"
  },
  "http": {
    "request_headers": {
      "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.82 Safari/537.36"
    }
  },
  "$schema": "/analytics/pref_diff/1.0.0",
  "user_id": 1,
  "before": "vector1",
  "after": "vector2",
  "bucketed_user_edit_count": "100-999 edits",
  "dt": "2021-03-18T18:03:24Z"
}

@MNeisler @ovasileva and I had a meeting this week in which we decided to drop the proposed "defaultState" field, in favor of only sending the initial and final state of the skin in each event.

We could, therefore, satisfy this task by adding an optional oldValue (or originalValue) property to the PrefUpdate schema rather than creating a new schema altogether.

Following on from the above, all of the issues described in T261842#6597536 would still exist and creating a new schema gives us the flexibility of tearing it down at the end of the project.

@Krinkle - see my comment in T261842#6759083. We would like to be able to determine the number of users that are opting in and out of desktop improvements on their own. Right now, PrefUpdate cannot distinguish between these users and the ones that are using the new version of vector by default (because we've changed the configuration on that wiki).

PrefUpdate only emitted when a preference is changed. When a new user is created or when a user is using a wiki day-to-day where a default was changed, this would not emit PrefUpdate just as much as it would not emit this new instrumentation, right? So by being emitted that seems like it is the signal you're looking for. If it changes "back" to the default then you know it is a choice and not the default-default since in that case there would not be an event.

Anyway, if you're mainly looking for the overall amount and have indeed no intention of correlating data from individual users, then maybe this type of low-level instrumentation is not needed at all. Rather, the periodic query you would run to analyse this data could be run on the database replicas itself? I may be missing something, but it seems like a daily query that records "current" overall counts by skin/skinversion and plotting that over time would give you exactly what you're looking for. Noting that in the database there is a clear distinction between implicit-default (null/no row) and a user having chosen/switched to something that is or was the default at some point (concrate value).

PrefUpdate only emitted when a preference is changed. When a new user is created or when a user is using a wiki day-to-day where a default was changed, this would not emit PrefUpdate just as much as it would not emit this new instrumentation, right? So by being emitted that seems like it is the signal you're looking for. If it changes "back" to the default then you know it is a choice and not the default-default since in that case there would not be an event.

This is correct - the PrefUpdate instrumentation is designed to only send out events when user initiated preference changes occur [1]. I think there may have been confusion regarding this fact. From my understanding, the PrefUpdate schema/instrumentation isn't currently meeting our needs because it isn't sending the before state of a preference change (e.g. we are interested in knowing that a user went from latest Vector to Monobook) which is info that the PrefUpdate instrumentation doesn't currently send. cc @MNeisler if there are any corrections or additional needs that I missed here

Anyway, if you're mainly looking for the overall amount and have indeed no intention of correlating data from individual users, then maybe this type of low-level instrumentation is not needed at all. Rather, the periodic query you would run to analyse this data could be run on the database replicas itself? I may be missing something, but it seems like a daily query that records "current" overall counts by skin/skinversion and plotting that over time would give you exactly what you're looking for. Noting that in the database there is a clear distinction between implicit-default (null/no row) and a user having chosen/switched to something that is or was the default at some point (concrate value).

I'll defer to @MNeisler on the periodic query question although I will add that the VectorSkinVersion user property does not currently behave like how other user properties behave. Other user property records (at least ones I've seen) are added to the database when they differ from their default value and get removed when set to their default value. The VectorSkinVersion record is created in the database when users who don't have that record change their preference but also when new user accounts are created [2] which makes it difficult to differentiate those who opted-in to latest Vector via a user preference change vs. someone whose account was created and defaulted to latest Vector just by looking at the database (although as I noted earlier, this distinction should actually be possible with the PrefUpdate instrumentation in theory).

@MNeisler Timo also asked on the new proposed schema patch whether a hash of the user ID was sufficient for your analysis (e.g. a user ID of 1 would look like c4ca4238a0b923820dcc509a6f75849b) or whether you were planning on joining or correlating the real user ID with other sources of information and would therefore require the real user ID.

[1] https://github.com/wikimedia/mediawiki-extensions-WikimediaEvents/blob/master/includes/PrefUpdateInstrumentation.php#L119-L121
[2] https://github.com/wikimedia/Vector/blob/a425c942a796464e6f3ce428509d95ed88304e62/includes/Hooks.php#L220

This is correct - the PrefUpdate instrumentation is designed to only send out events when user initiated preference changes occur [1]. I think there may have been confusion regarding this fact. From my understanding, the PrefUpdate schema/instrumentation isn't currently meeting our needs because it isn't sending the before state of a preference change (e.g. we are interested in knowing that a user went from latest Vector to Monobook) which is info that the PrefUpdate instrumentation doesn't currently send. cc @MNeisler if there are any corrections or additional needs that I missed here.

Yes, if we are just interested in tracking the total number of user-initiated selections for a specific desktop skin then the current PrefUpdate instrumentation should be sufficient. However, since there are more than two possible desktop skin preferences and a default value that can change depending on the user type, wiki and current configuration, it's difficult to determine the state of the user's preference before the switch. We can't assume that a selection for the legacy vector meant that the user is switching from the latest vector. For example, they could have switched from monobook and it's also possible a user made several desktop preference changes before the recorded switch. The inclusion of an initial state field along will help provide some context about the switch that will be useful in the analysis.

I'll defer to @MNeisler on the periodic query question although I will add that the VectorSkinVersion user property does not currently behave like how other user properties behave. Other user property records (at least ones I've seen) are added to the database when they differ from their default value and get removed when set to their default value. The VectorSkinVersion record is created in the database when users who don't have that record change their preference but also when new user accounts are created [2] which makes it difficult to differentiate those who opted-in to latest Vector via a user preference change vs. someone whose account was created and defaulted to latest Vector just by looking at the database (although as I noted earlier, this distinction should actually be possible with the PrefUpdate instrumentation in theory).

I agree that a periodic query using the replicas could let us know the overall counts for each skin preference over time but per @nray's comments, I'm not sure it would only show us users that explicitly changed their preferences given how the VectorSkinVersion property behaves. If we confirm it does, it would still have the same drawback that we have with the current PrefUpdate Instrumentation (unknown initial state of skin),

@MNeisler Timo also asked on the new proposed schema patch whether a hash of the user ID was sufficient for your analysis (e.g. a user ID of 1 would look like c4ca4238a0b923820dcc509a6f75849b) or whether you were planning on joining or correlating the real user ID with other sources of information and would therefore require the real user ID.

Yes, a hash for the user ID would be sufficient. No join should be needed for the analysis.

Update on the status of this ticket:

  • Per the title and description on this ticket, a new schema with the desired fields and instrumentation was created:
  • There were concerns raised in code review about the use of the user ID field and a recommendation to hash it using a secret that is generated on a deployment server using MWCryptHKDF::generateHex(32) from eval.php and set in wmf-config/PrivateSettings . A hashed user ID would be sufficient for our data needs per T261842#6939003.
  • I don't currently have access to deployment servers and therefore can't generate the aforementioned secret. I can apply to gain access, however, given the importance of this ticket, another options is for another dev on the team who has access to deployment servers to generate the secret.

I will inquire more on slack. I am moving this to blocked until a strategy for this ticket is fleshed out.

  • There were concerns raised in code review about the use of the user ID field and a recommendation to hash it using a secret that is generated on a deployment server using MWCryptHKDF::generateHex(32) from eval.php and set in wmf-config/PrivateSettings . A hashed user ID would be sufficient for our data needs per T261842#6939003.

Per your thread on Slack, AIUI you'll need to add a config variable to MediaWiki-extensions-WikimediaEvents, e.g. $wgWMEVectorPrefDiffSalt and use it where you're currently using $wgSecretKey. If you're happy with the suggested name, then we can ask in #wikimedia-sre to get the value generated and the variable added to the private repo.

Per your thread on Slack, AIUI you'll need to add a config variable to MediaWiki-extensions-WikimediaEvents, e.g. $wgWMEVectorPrefDiffSalt and use it where you're currently using $wgSecretKey. If you're happy with the suggested name, then we can ask in #wikimedia-sre to get the value generated and the variable added to the private repo.

Thank you @phuedx, I'll do that today

@phuedx https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/668530 has been updated to handle the private salt. I'm putting this back in code review for your review

Change 668529 merged by jenkins-bot:

[schemas/event/secondary@master] Add new analytics/pref_diff schema

https://gerrit.wikimedia.org/r/668529

nray reassigned this task from Edtadros to cjming.
nray reassigned this task from cjming to phuedx.
nray added a subscriber: Edtadros.
nray added a subscriber: cjming.

Change 668530 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] Add VectorPrefDiffInstrumentation to track all skin changes involving Vector

https://gerrit.wikimedia.org/r/668530

I moved this into Blocked on Others but it's really blocked on… us. We need to generate, configure, and deploy a value for $wgWMEVectorPrefDiffSalt. Regrettably, I haven't yet had time to do this.

Mentioned in SAL (#wikimedia-operations) [2021-04-08T18:51:35Z] <phuedx@deploy1002> Synchronized private/PrivateSettings.php: PrivateSettings: Add value for (T261842) (duration: 01m 06s)

Mentioned in SAL (#wikimedia-operations) [2021-04-08T18:52:57Z] <phuedx> phuedx@deploy1002 Synchronized private/PrivateSettings.php: PrivateSettings: Add value for $wgWMEVectorPrefDiffSalt (T261842)

@MNeisler - looks like this is deployed and ready for QA

The new schema and instrumentation have now been deployed (thank you @phuedx for deploying!) . I believe additions to config (involving wgEventStreams and wgEventLoggingStreamNames) are necessary in order to see the events though which is why I've moved this back to doing

nray removed nray as the assignee of this task.Thu, Apr 15, 5:30 AM

Config patches have been created, I'll plan to backport Thursday

Change 679613 had a related patch set uploaded (by Nray; author: Nray):

[operations/mediawiki-config@master] Add mediawiki.pref_diff stream to wgEventLoggingStreamNames/wgEventStreams

https://gerrit.wikimedia.org/r/679613

Change 679614 had a related patch set uploaded (by Nray; author: Nray):

[operations/mediawiki-config@master] Add $wgWMEVectorPrefDiffSalt to private/readme

https://gerrit.wikimedia.org/r/679614

Change 679613 merged by jenkins-bot:

[operations/mediawiki-config@master] Add mediawiki.pref_diff stream to wgEventLoggingStreamNames/wgEventStreams

https://gerrit.wikimedia.org/r/679613

Mentioned in SAL (#wikimedia-operations) [2021-04-15T18:32:01Z] <jdrewniak@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:679613|Add mediawiki.pref_diff stream to wgEventLoggingStreamNames/wgEventStreams (T261842)]] (duration: 01m 18s)

Change 679614 merged by jenkins-bot:

[operations/mediawiki-config@master] Add $wgWMEVectorPrefDiffSalt to private/readme

https://gerrit.wikimedia.org/r/679614

Mentioned in SAL (#wikimedia-operations) [2021-04-15T18:39:46Z] <jdrewniak@deploy1002> Synchronized private/readme.php: Config: [[gerrit:679614|Add $wgWMEVectorPrefDiffSalt to private/readme (T261842)]] (duration: 01m 08s)

@MNeisler As I mentioned in slack I believe all the config necessary to see the opt-in/opt-outs has been setup and deployed. I think it will be presented to you either as mediawiki.pref_diff or analytics/pref_diff (if you don't see anything with the name "pref_diff", that's probably a bad sign

I confirmed that events for this schema are logged and stored in eventlogging data as event.mediawiki_pref_diff.

Leaving this task open as I still need to do a more thorough QA check next week but based on an initial check of the data it appears that events are logging as expected. Thanks @nray and @phuedx for your help getting this deployed!