https://meta.wikimedia.org/wiki/Schema:MobileWikiAppNotificationInteraction

Remove icon (or ignore); will need to refine device level notifications flag.

Move device level opt in to Device state table.

If possible, it would be helpful to also track the following:

• Interactions with push notifications / notification banners while the app is open/active
• Destinations from the 'more' action sheet (eg. swipe from right to left on a notification cell and then tap on 'more' and then track where folks tap from there)

Schema design will be based on @cmadeo's definitions/descriptions documents:
Supporting notifications on the iOS app
Notifications on iOS: Notification Mapping v2
Also for reference, see android_notification_interaction schema

Change 747967 had a related patch set uploaded (by Sharvaniharan; author: Sharvaniharan):

[schemas/event/secondary@master] MEP schema for IOS Notification Interaction

https://gerrit.wikimedia.org/r/747967

Note: Thanks to @Sharvaniharan for assisting here and uploading the schema doc to MEP on Gerrit, we agreed that going forward it makes sense to have an engineer who has regular access to Gerrit do this step since it can be a bit tricky if there are any issues with formatting, etc. She also worked on instrumenting the Android notification interaction schema so feel free to contact her if you run into issues.

For reference, below is the schema configuration:

0 - Mark as read and archive  
1 - Primary action (as specified by Echo API)  
2 - Secondary action (as specified by Echo API)  
3 - Link clicked (as specified by Echo API)

In cases where user has selected multiple notifications and has marked all as read (and archived),
a unique identifier is generated for the set and included with the event sent for each of the notifications.
This enables us to analyze usage of the multi-select feature

Change 747993 had a related patch set uploaded (by Sharvaniharan; author: Sharvaniharan):

[operations/mediawiki-config@master] Add event stream config for ios.notification_interaction

https://gerrit.wikimedia.org/r/747993

Change 747967 merged by jenkins-bot:

[schemas/event/secondary@master] MEP schema for IOS Notification Interaction

https://gerrit.wikimedia.org/r/747967

Change 747993 merged by jenkins-bot:

[operations/mediawiki-config@master] Add event stream config for ios.notification_interaction

https://gerrit.wikimedia.org/r/747993

Mentioned in SAL (#wikimedia-operations) [2022-01-13T19:19:50Z] <taavi@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:747993|Add event stream config for ios.notification_interaction (T290920)]] (duration: 01m 13s)

Hi @SNowick_WMF @JMinor

I have a some questions around the event rank options before we pull this into Ready for Dev:

Mark as read and archive.

1. Do we also send this if the user marks a notification as unread?
2. More of an FYI, iOS doesn't have an archive feature.
3. We also have a "Mark all as read" option, where we ask the server at the various wikis to mark all their notifications as read. It's not associated with any individual notifications, so the selection token structure feels a bit odd with this setup. We could still do a local fetch to gather any unread notifications and use those as the individual notifications for the events when this button is tapped, but just thought I'd call it out.

Primary actions, secondary actions, and link clicked
I remember understanding this before but now I'm confused again - can we get in writing what each of these mean? As a reminder, these are the different tap areas that do things:

1. Tapping the notification cell. This takes user to the "primary" link. Link destinations are in this spreadsheet. This might not exactly match what the Echo API tells us - we try our best to create the spreadsheet link destinations first, but if it's an unrecognized notification type we use the links that come from the Echo API. Note tapping a cell also automatically marks the notification as read if needed. So should we send two events (primary and mark as read) for this action?
2. Swiping the cell and choosing the "mark as read/unread" button. This marks an individual cell as read/unread.
3. Swiping the cell and choosing the "More" menu. This pops up a list with other actions. Note this list also has some repeated destinations (like another mark as read/unread button, and another link to the primary destination from step 1). It also has a link to Notification Settings in the app.
4. Tapping Edit, selecting multiple cells, and choosing "mark and read" or "mark as unread" options from the contextual menu.
5. Tapping Edit, selecting "Mark All As Read".

Hi, hopefully the following will get us closer to addressing these details, if anything is unclear or needs further discussion we can hammer out details further. I did check with @Dbrant on how we do this on Android for verification on some of these as well.

Mark as read and archive.

1. We should check with Josh on whether we want to explicitly track Mark as Unread events, but in the case that we do we will need to make an additional `action_rank’ event and assign it a number between 4 and 9.
2. For a “Mark All as Read” event, if we want to just track the button click it we can either make it a singular event, making another additional explicit `action_rank’ event and assigning it a number or if we want to keep track of all the local unreads that we can locally track and assign the ‘group’ selection token to each message, we can do that instead. It makes more sense to me to just track “Mark All as Read” as a singular event but we may want to track each message that’s marked as read together. Will discuss this with @JMinor

Primary actions, secondary actions, and link clicked

1. It’s fine to use the Echo API link when our own isn’t possible, I’m assuming that will not happen too frequently, we may want to think about tracking when that is necessary (by adding a flag to the event in another column) if we think we need to. Only one event needs to be sent when the action_rank is 1,2 or 3, as a message being marked as read when clicked is implicit.
2. Swiping the cell ‘Mark as read’ should all be action_rank = 0, if the click is for ‘Mark as unread’ and we decide to track that, the event should be labeled with the corresponding action rank we decide on.
3. Even if a primary destination link click, or “Mark as Read” comes from swiping a cell or the More menu it should still be counted as the corresponding action_rank event. The end result of the click should determine what the event is labeled.
4. From Edit, if multiple messages are selected and marked as read each selection marked is an event (so multiple action_rank = 0 events). Since it’s a manual selection they can count as multiple events.
5. From Edit “Mark All as Read” events will be the same as #2 above.

Thanks @SNowick_WMF ! I think the biggest remaining question I have is how primary or secondary actions relate to link clicked. When do we send the link clicked event?

It makes more sense to me to just track “Mark All as Read” as a singular event

I agree, I think this should be an additional singular event rather than something that is triggered against an individual notification, because that lines up more with how it happens on the server-side. It could be beneficial to know which wikis the user is marking all as read on (if they have unread notifications on both EN and ES, we send the nuclear "mark all as read" call to both EN and ES Wikipedia, for example), but that's maybe not necessary information.

Per our sync:

• Mark all as read is out of scope for this event format. If we want to track use of mark all as read, lets create a specific event for it.
• Mark and read and unread will both be the same event. Basically the event will represent toggling a message between read states, and we won't have distinct read and unread counts.

Sync update regarding how the action_rank value works with tap destinations:

1. When tapping a cell, because it takes the user to both the primary* destination and quietly marks it as read, we will send two events, one with an action_rank value of 0 and another with an action_rank value of 1.
2. When swiping a cell, choosing More, and choosing an option, we will keep it very simple: just send the zero-indexed integer order as it appears in the action sheet options. Since mark as read/unread is always the first one*, we'll be sending an action_rank of 0. If there are five items in the list and Notification settings appears as the fifth item and it's tapped, we send an action_rank of 4. If there are three items in the list, Notification settings appears as the third item and it's tapped, we send an action_rank of 2.

*Because we are constructing a lot of these urls independent of the primary and secondary distinctions the API makes, we should do an audit to make sure our primary goes to the same destination (within the app) as the primary link from the API, and our secondary orderings in the More action sheet match the secondary ordering from the API. They should match, because the url construction logic was based on the spreadsheet, and the spreadsheet was based on the API/web destinations. If there's any discrepancy, let @JMinor or @cmadeo know.

@SNowick_WMF one last excellent question came up in Engineering sync - just a confirmation, for notification type, we are sending the type value that we get back from the notifications API (log in on EN and see example response here), correct? I ask because we also have a category property in that API and I wanted to make sure you didn't want it. It almost looks like an umbrella type for groups of notification types but I'm not sure what it's purpose is.

*Because we are constructing a lot of these urls independent of the primary and secondary distinctions the API makes, we should do an audit to make sure our primary goes to the same destination (within the app) as the primary link from the API, and our secondary orderings in the More action sheet match the secondary ordering from the API. They should match, because the url construction logic was based on the spreadsheet, and the spreadsheet was based on the API/web destinations. If there's any discrepancy, let @JMinor or @cmadeo know.

@JMinor @cmadeo Now that we are about to pick this up, I made another spreadsheet consisting of:

1. The actions from the API (if I was able to trigger the notification)
2. The swipe menu actions (based on Swipe Actions defined by design in a previous spreadsheet).
3. The detail screen actions (loosely based on mocks and swipe actions)

You'll notice there's many inconsistencies, so I want to confirm once more what we're sending as an action_rank value. According to the schema documentation:

action_rank:
  type: integer
  description: >
    0 - Mark as read and archive
    1 - Primary action (as specified by Echo API)
    2 - Secondary action (as specified by Echo API)
    3 - Link clicked (as specified by Echo API)

According to previous instruction, we should send the sort order of the swipe menu actions, and I'm guessing we treat the detail screen secondary actions area similarly. Using a User Talk message type as an example (first row on this spreadsheet), this will result in different action_rank values being sent for the same destination. Let me know if this is okay - if not, we may need to change the ordering of these actions to be more consistent.

@Tsevener thank you for the amazing spreadsheet. Re: Inconsistencies, would you have a preference from a technical standpoint for the ordering to be the same in the Swipe menu as in the detail view? I'm happy to update the spreadsheet with this if that's helpful?

Note: For analytics we will be using string values for event tracking, rather than the numeric code described above as action_rank. Hopefully this will let us sidestep the order/API code issues. I'm working on the schema doc now , will use Toni's spreadsheet and screenshots to illustrate the string values for each action. Will share doc when it's laid out but let me know if there are any questions before then. Thanks!

@Mazevedo, @Tsevener
Schema slide deck: here
Schema Spec Spreadsheet based on latest version of notification events.

If there are changes and adjustments please add them here so we can track, I will modify docs to reflect new values, etc.

Change 784293 had a related patch set uploaded (by Tsevener; author: Tsevener):

[schemas/event/secondary@master] Modify ios_notification_interaction schema

https://gerrit.wikimedia.org/r/784293

Change 784293 merged by Ottomata:

[schemas/event/secondary@master] Modify ios_notification_interaction schema

https://gerrit.wikimedia.org/r/784293

Mentioned in SAL (#wikimedia-analytics) [2022-04-25T20:09:13Z] <ottomata> dropping event.ios_notification_interaction hive table and data for backwards incompatible schema change in T290920

Hi @SNowick_WMF,

A few things came up from analytics code review that I wanted to clarify:

1. action should be a required field right? I figure there’s little use for you if we send it as null, but I just wanted to make sure.
2. There are a few action values that might be missing from the schema. Do you want to track when the user selects these too? If so, what values would you like for them?
   • Link to Getting Started page (seen on Welcome notification)
   • Link to Help:Login_notifications page (seen login-related notifications)
   • Link to Change Password page (seen on login-related notifications)
   • Tapped a link with a less common namespace (i.e. we constructed a link with a namespace that isn't User_talk, Talk, User, or Main, like "Wikipedia" namespace or something).
   • Tapped a generic link (we wouldn't know specifics beyond that, but maybe worthwhile to know they are digging deeper on notifications that do not have a custom treatment).

Hi thanks for checking on these -

1. Since the schema is for interactions, yes action should be a required field
2. Added values to Spec spreadsheet with definitions, as follows:

*For clarification - for item action = link_namespace - can we replace namespace with actual namespace value? If not it's ok to leave as is, lmk if you want this to be a specific value or if the generic 'namespace' is ok.

For clarification - for item action = link_namespace - can we replace namespace with actual namespace value? If not it's ok to leave as is, lmk if you want this to be a specific value or if the generic 'namespace' is ok.

@SNowick_WMF We could send whatever is in this PageNamespace enum of ours - would that work? We could send the integer value ("4"), or convert it to a string ("wikipedia"), or both the link_namespace indicator with the integer or string value combined "link_namespace-4" or "link_namespace-wikipedia". Let me know if you have a preference.

Yes, the PageNamespace values are useful, converting to the string that you've already defined from the integer is probably the best value to work with. So a user clicks a link to namespace integer = 9 should be action = link_mediawikiTalk, a user clicks a link to namespace integer = 11, the value will be action = link_templateTalk. That way we aren't converting values after data is pulled from schema.

These events should be available in the next TestFlight build. I'm not sure why there's still a Patch-For-Review tag on this task - it looks like all the Gerrit patches have been merged.

We're seeing a small amount of data in the new ios_notification_interaction schema. Looks like a healthy variety of events are being counted. Users who appear in ios_notification_interaction are also showing values for inbox_count and device_level_enabled in MobileWikiAppiOSUserHistory.

Will re-check when more data is collected and then again when release goes into production.

Data is validated - see Superset Dashboard for notification event frequency charts/data.