Page MenuHomePhabricator

Log when a user enables/disables IP Info
Closed, ResolvedPublic2 Estimated Story Points

Description

Background

Following on from T296415, we'd like to know how many users are enabling or disabling IP Info

AC
  • When I enable IP Info, an event should be logged
    • The event_action property should be "enable_ipinfo"
    • The event_context is page and the event_source is special_preferences
  • When I disable IP Info, an event should be logged
    • The event_action property should be "disable_ipinfo"
    • The event_context is page and the event_source is special_preferences
Notes
  1. See the current.yaml files in the subdirectories in https://gerrit.wikimedia.org/r/plugins/gitiles/schemas/event/secondary/+/refs/heads/master/jsonschema/analytics/mediawiki for numerous examples of JSON Schema schemas that are used in production
  2. wt:Event Platform/Instrumentation How To is a step-by-step guide to creating a creating a schema and validating an event with it
  3. https://www.mediawiki.org/wiki/MediaWiki-Docker/Configuration_recipes/EventLogging is a step-by-step guide to setting up your local development environment

Event Timeline

@jwang: While writing this task up, I remembered that the user can agree to IP Info's terms of use via both a special UI in the infobox on Special:Contributions and Special:Preferences. In the former case, I've suggested that the event_source property be "special_preferences"; in the latter, "accordion".

ARamirez_WMF renamed this task from Log when a user enables or disables IP Info to Log when a user enables or disables IP Info [S].Nov 30 2021, 5:38 PM
ARamirez_WMF renamed this task from Log when a user enables or disables IP Info [S] to Log when a user enables or disables IP Info.Feb 15 2022, 6:49 PM
ARamirez_WMF set the point value for this task to 2.
STran renamed this task from Log when a user enables or disables IP Info to Log when a user enables IP Info from the infobox.Mar 15 2022, 8:19 AM
STran updated the task description. (Show Details)

@jwang Are you the right person to ask? We already track agreeing to the use agreement as accept_disclaimer from T296477: Log when the user sees and bounces from the terms of use agreement in the infobox. Do we want to double up?

Hi @STran, Do you mean the accept_disclaimer and check_iagree are same event and it is redundant to capture them in two schemas? Based on the prototype, the user flow is disclaimer shown-> check_iagree (accept_disclaimer) -> enable_ipinfo.

It's better that above all actions can be captured in one schema, so that we have the sequence of the events to understand the opt-in journey better. If not, then we need to have a unique key which can join the schema for check_iagree and the schema for enable_ipinfo. If two schemas have the unique key to join, it's OK to just record the event in one of the schemas.

image.png (480×1 px, 72 KB)

@jwang yes accept_disclaimer and check_iagree are redundant. Could you clarify what you mean by the schemas? Is this something that's already set? I understand schemas to be the event schemas. Just to be clear, am I correct in understanding that I don't have to log check_iagree in addition to accept_disclaimer?

Additionally, there isn't any pathway where the user accepts the agreement and doesn't also enable ip info:

  1. on Special:Preferences, the user can only accept the agreement if they've already asked to enable ip info. Accepting the agreement enables ip info at the same time.
  2. on Special:Contributions, the user is only shown the infobox with the agreement if they've already asked to enable ip info. Accepting the agreement enables ip info at the same time.

We would be logging both events (accept_disclaimer, enable_ipinfo ) at the same time. Is this redundancy ok?

@STran , thanks for clarification.

I don't have to log check_iagree in addition to accept_disclaimer?

Yes.

We would be logging both events (accept_disclaimer, enable_ipinfo ) at the same time. Is this redundancy ok?

It's OK.

@jwang I'm so sorry I just finished a chat with @phuedx and he pointed out this reference: https://docs.google.com/spreadsheets/d/16S5fqAEDz3A-Lu0rvtkiIgyeP_00ZX8qjz6GhH9fIlk/edit#gid=0

It looks like enable_ipinfo was originally meant to refer to the first checkbox here:

image.png (134×578 px, 16 KB)

and it doesn't look like we meant to track a final "the user can now see ip info" event (L51-2 in the spreadsheet?). Could you confirm which meaning of enable_ipinfo we should use moving forward? The checkbox or the final event? Did we want both?

I am confusing now. Does it mean both of below two interfaces can equally enable IP info? Which one is the final step? We need to record the final step of enabling IP info. If both of them can be the final step, we need to record both.
Interface 1:

image.png (134×578 px, 16 KB)

Interface 2:
image.png (480×1 px, 72 KB)

Does it mean both of below two interfaces can equally enable IP info?

Yes

Which one is the final step?

Checking the "I agree to use the information for my patrolling and anti-vandalism work"
This can be done from either interface 1 or 2.

We need to record the final step of enabling IP info.

No problem. Do we still need to record when "Show the IP Information tool on Special:Contributions page for IP addresses." is checked or unchecked?

Do we still need to record when "Show the IP Information tool on Special:Contributions page for IP addresses." is checked or unchecked?

If it's not checked, users won't see the IP info feature, right? If so, we need to record it to exclude users who unchecks it in new feature analysis.

Thanks @jwang

Do we still need to record when "Show the IP Information tool on Special:Contributions page for IP addresses." is checked or unchecked?

If it's not checked, users won't see the IP info feature, right? If so, we need to record it to exclude users who unchecks it in new feature analysis.

If I understand correctly, we only need to record whether this checkbox is checked/unchecked in cases where it changes whether the IPInfo tool is enabled for the user. (It is possible for the tool to remain disabled even if the user checks/unchecks the box, e.g. if they do so without accepting the terms of use. My understanding is that we don't need to record when this happens?)

@STran Here are some thoughts about how we could distinguish between the infobox and Special:Preferences when we log the event_source.

  1. Don't do it if we don't need to. @jwang Do we actually need to record whether the user enabled the tool from interface 1 or interface 2 (T296472#7794010)? If not, then perhaps we could just always record the event_source as Special:Preferences.
  1. Infer it from $modifiedOptions in PreferencesHandler::onSaveUserOptions. As @STran pointed out, this array only contains one option for the infobox, but a lot more options from Special:Preferences. As long as we comment it well and remember to update it if we ever add more options to the infobox form, or add another interface, I think this should be fine. (It might be better to check for the length of the $modifiedOptions array rather than looking for the presence of a specific option that we expect to see when saving Special:Preferences, since the available preferences can be modified by the onGetPreferences hook, so I don't think any specific preferences are guaranteed to be there. They could in theory all be deleted, but I think that's vanishingly unlikely.)
  1. Log somewhere specific to each interface. We already track the enable/disable from the infobox on fulfillment of the promise returned by Api.saveOption. We could use the onPreferencesFormPreSave hook to log enable/disable from Special:Preferences. This happens a bit before the onSaveUserOptions hook, so we could have some inaccuracies in edge-cases where something goes wrong between the two hooks running; however something could already go wrong after onSaveUserOptions, so this wouldn't be adding much more risk.

NB I dismissed another approach: checking RequestContext::getMain from PreferencesHandler::onSaveUserOptions. This would tell us what we need to know, but is a hack that we shouldn't really use.

@Tchanders, thanks for pinging me

Don't do it if we don't need to. @jwang Do we actually need to record whether the user enabled the tool from interface 1 or interface 2 (T296472#7794010)? If not, then perhaps we could just always record the event_source as Special:Preferences.

Where the clicks are from is not a question asked in the measure question list (T292803). But I deferred to @Niharika in case she wants to have it measured.

@Tchanders

My understanding is that we don't need to record when this happens?)

Agree.

@Tchanders, thanks for pinging me

Don't do it if we don't need to. @jwang Do we actually need to record whether the user enabled the tool from interface 1 or interface 2 (T296472#7794010)? If not, then perhaps we could just always record the event_source as Special:Preferences.

Where the clicks are from is not a question asked in the measure question list (T292803). But I deferred to @Niharika in case she wants to have it measured.

I concur. It does not matter where IP Info is enabled from.

Thanks @Niharika, @jwang

@STran - should this task be re-merged with T303796?

STran renamed this task from Log when a user enables IP Info from the infobox to Log when a user enables/disables IP Info.Mar 25 2022, 2:32 AM
STran updated the task description. (Show Details)

I'm being bold and splitting off the enable/disable IPInfo tracking from the check/uncheck agreement events. My reasoning for this is:

  • we agreed to lie to the schema and are going to log the event PHP-side via the onSaveUserOptions hook
  • whoever is doing the work around checking/unchecking gets to decide whether they want to also do that from the php side or if they want to do it via js
  • we agreed that check_iagree was redundant because we are already tracking accept_disclaimer

Change 773672 had a related patch set uploaded (by STran; author: STran):

[mediawiki/extensions/IPInfo@master] Add EventLogging for enable/disable IPInfo events

https://gerrit.wikimedia.org/r/773672

Change 773672 merged by jenkins-bot:

[mediawiki/extensions/IPInfo@master] Add EventLogging for enable/disable IPInfo events

https://gerrit.wikimedia.org/r/773672

@STran those events are not showing in Beta. I do see events on Special contribution, non is showing on special preference page