I suspect no additional instrumentation will be needed.
A page-level image suggestion is currently a single event. Section-level image suggestions will probable be multiple events, that the Echo UI will simply group/present as 1 (and a slight different in their payload/data)

Discerning between section & page suggestions can probably be done by looking at their data (one will probably include details about the section, the other won't), and figuring out whether it's a combination of suggestions can probably be done by grouping per user & page.

More details to follow after having completed T330931.

Moving to blocked until T330931 is complete.

In T331966#8732531, @matthiasmullie wrote:

I suspect no additional instrumentation will be needed.
A page-level image suggestion is currently a single event. Section-level image suggestions will probable be multiple events, that the Echo UI will simply group/present as 1 (and a slight different in their payload/data)

Discerning between section & page suggestions can probably be done by looking at their data (one will probably include details about the section, the other won't), and figuring out whether it's a combination of suggestions can probably be done by grouping per user & page.

More details to follow after having completed T330931.

@cchen can you confirm that this will work? thank you!

In T331966#8732531, @matthiasmullie wrote:

I suspect no additional instrumentation will be needed.
A page-level image suggestion is currently a single event. Section-level image suggestions will probable be multiple events, that the Echo UI will simply group/present as 1 (and a slight different in their payload/data)

Discerning between section & page suggestions can probably be done by looking at their data (one will probably include details about the section, the other won't), and figuring out whether it's a combination of suggestions can probably be done by grouping per user & page.

More details to follow after having completed T330931.

So for section-level and combined image suggestions, there will be multiple entries in the notification table, but with the same notification event_id right?

Will the details about the section be stored in the same place as where we store the image file name?

@matthiasmullie following up on the above, thanks!

Hard to tell for sure without having even begun the work, but probably yes on both of those.

Note: leaving this open until T330931 has been completed to make sure above assumption are correct (or do what is needed otherwise)

Now that this has been implemented, here are the details for how these notifications end up being stored:

• Every suggestion (for both page & section) is an individual notification in echo_event with event_type = image-suggestions
• The difference between a page or section suggestion is in the JSON data in event_extra:
  • Section suggestions will have a section-heading key with a string value (the section name)
  • Page suggestions will have a section-heading key with a null value (for suggestions from now on), or no such key at all (for page suggestions prior to having introduced section suggestions)
• Combined suggestions (more than one suggestion for a page, e.g. page suggestion + some section suggestions) are composed of multiple echo_event entries: echo_events with the same event_agent_id (the user receiving the notification) and event_page_id (the page the notification is for) will show as one bundled suggestion that can be expanded to show the individual details

@cchen Does above make sense? Let me know if you need anything else, otherwise you can close the ticket.

(Moving to blocked until @cchen closes, or more detail is needed)

@matthiasmullie thanks for the details! to confirm, for combined suggestions, multiple events entries have the same event_id as well right?

No (I was wrong when I acknowledged that before); they're all individual echo_event entries with unique event_id values, but their event_agent_id & event_page_id will be the same.

E.g.:

+----------+-------------------+---------------+----------------+----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+---------------+
| event_id | event_type        | event_variant | event_agent_id | event_agent_ip | event_extra                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | event_page_id | event_deleted |
+----------+-------------------+---------------+----------------+----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+---------------+
| 47122973 | image-suggestions | NULL          |        1683491 | NULL           | a:2:{s:9:"media-url";s:48:"//commons.wikimedia.org/wiki/File:Indicatori.png";s:15:"section-heading";N;}                                                                                                                                                                                                                                                                                                                                                                                     |            20 |             0 |
| 47122972 | image-suggestions | NULL          |        1683491 | NULL           | a:2:{s:9:"media-url";s:60:"//commons.wikimedia.org/wiki/File:Speech_bubbles_clipart.png";s:15:"section-heading";s:37:"Предмет_лингвистики";}                                                                                                                                                                                                                                                                                                                                                |            20 |             0 |
| 47122963 | image-suggestions | NULL          |        1683491 | NULL           | a:2:{s:9:"media-url";s:63:"//commons.wikimedia.org/wiki/File:Gitanas_Nauseda_(cropped).png";s:15:"section-heading";s:41:"Государственный_строй";}                                                                                                                                                                                                                                                                                                                                           |             7 |             0 |
| 47122962 | image-suggestions | NULL          |        1683491 | NULL           | a:2:{s:9:"media-url";s:74:"//commons.wikimedia.org/wiki/File:Thermal_ionization_mass_spectrometer.jpg";s:15:"section-heading";N;}                                                                                                                                                                                                                                                                                                                                                           |           169 |             0 |
+----------+-------------------+---------------+----------------+----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+---------------+

The first (event_id = 47122973) and fourth (event_id = 47122962) are page-level suggestions: their section-heading (in event_extra) value is null (s:15:"section-heading";N;)
Second (event_id = 47122972) and third (event_id = 47122963) are section-level suggestions; they have a string value (the section name) for section-heading

The first (event_id = 47122973) and second (event_id = 47122972) are combined, because they're for the same page (event_page_id = 20) and user (event_agent_id = 1683491)
The third (event_id = 47122963) and fourth (event_id = 47122962) are individual notifications; there are no other entries with the same event_page_id and event_agent_id.

Thanks for the examples. The queries i used before for the notification data is

SELECT 
   notification_event,
   notification_user,
   notification_timestamp,
   notification_read_timestamp,
   event_extra,
   event_page_id
FROM echo_notification
JOIN echo_event on event_id = notification_event
WHERE notification_timestamp >= {start_timestamp}
  AND notification_timestamp < {end_timestamp}
  AND event_type = 'image-suggestions'

in this case, there will also be two notification events in the echo_notification table. In order to look for the number of notifications sent and read, I will need to use event_page_id and event_agent_id and maybe the timestamp as well. Will we send multiple notifications to one user with different suggestions for the same page? Because I looked at notification_timestamp, the combined events don't have the same notification timestamp the first is 20230703130551 and the second is 20230703130555.

All notifications for the same event_page_id & same event_agent_id are bundled. That's also explicitly the basis on which they're being bundled by (so even if we were to send multiple series of notifications for the same page, they'd still end up being bundled)

So basically, a GROUP BY event_page_id, event_agent_id should do.
Or you can GROUP BY echo_notification.notification_bundle_hash, which will also be the same for all notifications in the same bundle.

Note though that "notification_read_timestamp" is only recorded when the notification is clicked, and in the case of a bundle, that's going to be each individual one.