Page MenuHomePhabricator

[REQUEST] Metrics for push-notifications service
Closed, DeclinedPublic

Description

What's requested:
We (product-infrastructure) are in the process of deploying our push-notifications project. Its purpose is to provide API endpoints so wikipedia website and apps can send push notifications to subscribed users. We already have a baseline of operational metrics provided by service-runner as described in this grafana template. On top of that we want to define a set of metrics from a product perspective.

Why it's requested:
We would like to be able to demonstrate the value of the project with the metrics collected.

When it's requested:
We are planning the deployment of the service to production and the timeline is about 2-4 weeks (starting week of July 6 2020).
I am not sure when we are going to start sending push notifications to actual clients so we can collect production metrics but it would be great to have the functionality in place when we launch to prod.

Other helpful information:

Event Timeline

Hi @Jgiannelos, stopping by to confirm that we (the Product Analytics team) have seen this, and will triage it in our upcoming board refinement meeting on July 7.

LGoto triaged this task as Medium priority.Jul 14 2020, 5:13 PM
LGoto moved this task from Triage to Needs Investigation on the Product-Analytics board.

@SNowick_WMF This ticket is to put together a list of things we want to collect, where and when. Let's get together with @JMinor and hash it out.

Update:
Instrumented in next update:

  • dismiss rates for notifications
  • We can see open rates, (CTR if there is a link). Notification itself is a link or can have actions at the bottom that can go to different destinations
  • Send/deliver rates, do we have a way to see those metrics? Sum of clicks + dismisses = all impressions
SNowick_WMF lowered the priority of this task from Medium to Low.Feb 9 2021, 6:07 PM
SNowick_WMF moved this task from Next 2 weeks to Blocked on the Product-Analytics (Kanban) board.
kzimmerman added a subscriber: kzimmerman.

There are currently no plans to use the Push Notifications service, marking this as declined

Hi @kzimmerman, I think there's a misunderstanding, in that the service is very much in use on Android and development will begin on push notifications on iOS in the next couple weeks with target release in Jun.

It may still be valid to close this if we don't plan or need to do service level metrics. But the service is in active and growing use.

Per our team meeting, we'll create a fresh ticket for client level/iOS metrics rather than reopen this one.

To follow-up: the Push Notifications service, as Josh pointed out, is being used, but we have not heard of analytics needs for service-level metrics (beyond what can be tracked in Grafana for monitoring purposes). Keeping this task declined.