Page MenuHomePhabricator

Numerous Notification Tracking Graphs Stopped Working at End of 2015
Closed, ResolvedPublic

Description

A number of the graphs on the English Wikipedia Editor Engagement page, Notifications tab, stopped working at the end of 2015. These include:

  • Daily changes in email notification preferences by category
  • Daily changes in Web notification preferences by category
  • Daily changes in badge notification preferences
  • Daily clicks on notifications by user type
  • Daily notification views by user type
  • Daily clicks on notifications by source
  • Daily notification views by source
  • Daily clicks on notifications by category
  • Daily notification views by category

I'm assuming there's some common thread here that, once untangled, will fix these all. If that's not true and fixing each graph is a separate job of work -- or if fixing that one thing is itself a big job -- we should have a conversation about prioritizing the graph or graphs we think are the most useful.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 7 2016, 11:10 PM

Hi @jmatazzoni, these charts were designed a while ago when my team was involved in generating analytics on demand for specific features/products. They are not actively maintained. You may want to talk to @Nuria in Analytics if you need help visualizing custom data from the DBs or EventLogging tables, I don't know if we have a standard solution for this purpose (cc'ing Analytics and removing Research).

DarTar edited projects, added Analytics; removed Research.May 9 2016, 9:44 PM
DarTar removed a subscriber: DarTar.
Nuria added a comment.May 16 2016, 4:52 PM

@jmatazzoni : if you are interested on this data you can help us migrate the scripts that harvest it. Let us know if you are interested in collaborating

@Nuria writes:

if you are interested on this data you can help us migrate the scripts that harvest it. Let us know if you are interested in collaborating...

Thanks for your offer. I was hoping that there was one thing that needed fixing that would fix all of these graphs. But if they need to be attended to individually, and I have to choose, I'd break these down as follows:

USEFUL
These are graphs that I think can be helpful to us in making ongoing decisions.

  • Daily clicks on notifications by category
  • Daily notification views by category
  • Daily clicks on notifications by source
  • Daily notification views by source
  • Notifications by days taken to read

NOT USEFUL AT THIS TIME
Many of the data types below are ones that I can imagine needing to look up at some point in time, but these are not figures we need to track over time.

  • Daily notification views by user type
  • Daily clicks on notifications by user type
  • Daily changes in badge notification preferences
  • Daily changes in email notification preferences by category
  • Daily changes in Web notification preferences by category
  • Daily changes in email notification frequency preferences

What do we need to do to get these graphs up and running again?

Actually from what I can see ( @Milimetric please correct me if I am wrong) these metrics are already migrated to use reportupdater and
it seems that Neil has worked on the data @jmatazzoni is interested in, maybe @Neil_P._Quinn_WMF can chime in here?

See: https://github.com/wikimedia/analytics-limn-ee-data/commits/master/ee/days_to_read.sql

@Nuria, I have tweaked the SQL in that repo and it does use reportupdater, but it only contains the queries for the bottom two graphs on the page Joe is talking about. Those two are working fine.

The other graphs are in https://github.com/wikimedia/limn-editor-engagement-data, and I'm pretty sure they don't use reportupdater. I'm starting to work on dashboarding right now (T132984), so I can put migrating them on my backlog.

Nuria added a comment.May 24 2016, 3:45 PM

@Neil_P._Quinn_WMF Ah, I see, Thanks for looking into it, let us know if you need any help.

I just like to point out oout that overall we discourage metrics that are "absolute" in nature (like "number of clicks", "number of notifications") but if you feel those work well for your team please go ahead and migrate them. Absolute metrics vary a lot due to factors outside your control. We prefer using ratios or more descriptive metrics when possible: Like "ratio of daily notifications versus pageviews" insteads of "daily notifications".

Jdforrester-WMF triaged this task as Medium priority.Jul 26 2016, 6:08 PM
Jdforrester-WMF lowered the priority of this task from Medium to Low.
Jdforrester-WMF moved this task from Backlog to Next up on the Contributors-Analysis board.
nshahquinn-wmf raised the priority of this task from Low to Medium.Aug 9 2016, 5:49 PM
jmatazzoni closed this task as Resolved.Jan 5 2017, 10:54 PM
nshahquinn-wmf raised the priority of this task from Medium to Needs Triage.Mar 30 2018, 10:34 AM
nshahquinn-wmf moved this task from Backlog to Done on the Contributors-Analysis board.