Page MenuHomePhabricator

Investigate old Echo EventLogging data for TheWikipediaLibrary notification
Closed, ResolvedPublicSpike

Description

The TheWikipediaLibrary extension sends users a single notification when they cross the threshold of 500 edits and 6 months activity. @sjvipin hasn't received the notification on his volunteer account പയ്യൻ despite 21 months of activity and 508 edits. This is leading to some anxiety that the notification has stopped working for some reason. We're still investigating that specific case, but regardless this raises the issue that we should have some monitoring in place to ensure that the TWL notification is behaving as expected.

Echo data in EventLogging was recently decommissioned (T344167), which we missed. Although we weren't using it the data may have been helpful for us. We'd like to investigate that data to see if we should emulate it in Event Platform, or if brand new logging will be required.

With the older EventLogging data, could we find daily counts for the number of TheWikipediaLibrary notifications sent to users?

Event Timeline

Restricted Application changed the subtype of this task from "Task" to "Spike". · View Herald TranscriptSep 21 2023, 2:09 PM
Restricted Application added a subscriber: Sadads. · View Herald Transcript

New daily logins for the library look fine, so I don't think this is a total outage of TWL notifications.

Screenshot 2023-09-21 at 16.19.56.png (2,316×1,412 px, 332 KB)

@Samwalton9-WMF

I have analysed the logs, and there is no outage of TWL notifications. Although, the overall number notifications sent has decreased over time, there hasn't been any significant change in the last few weeks/months.

download.png (1,500×1,000 px, 143 KB)

Average daily number of notifications sent by month

yearmonthnotifications_sent
20221656
20222433
20223164
20224113
2022591
2022672
2022765
2022858
2022950
20221044
20221146
20221237
2023140
2023237
2023334
2023433
2023531
2023630
2023732
2023830
2023926
20231028

Also, I realized this after our conversation, the Echo EventLogging instrumentation was for user interactions of Echo notifications, but the actual Echo notification logs are available in the MediaWIki extension databases, echo_event and echo_notification, which are in use.

For regular monitoring, a Superset dashboard can be setup. Unless we need more data related how users are interacting with TWL notifications (apart notification_read_timestamp which is available in echo_notification table), we don't need any new instrumentation or data collection.

Thanks @KCVelaga_WMF! This confirms that we haven't had any sudden outage or bug, so we can go looking elsewhere. A slow decrease to a steady state is expected.

I think as long as your queries are saved somewhere (maybe here in Phab would be helpful), we can check on this if we're ever concerned but I think we need to set up dashboards.

@Samwalton9-WMF Sure, let's talk about the dashboard when we meet next. Here is the notebook and the query (P52969).