Page MenuHomePhabricator

Add logging for measuring impact of our work on improving the mobile block messages
Closed, ResolvedPublic3 Estimated Story PointsSep 22 2020

Description

Goal

This ticket relates to T189717: Better handle block reasons on mobile (specifically templates and HTML comments) which has some more context. We are making significant improvements to block messages on mobile. This has been brought up time and again as a problem for our end users who unexpectedly find themselves blocked.
This is important work and we want to make sure we can measure and highlight the impact we're having by doing this work.

Acceptance criteria:
  • Log the following:
    • How many times does the mobile block drawer display for end users?
    • How many times does a 'call to action' display (other than 'OK')?
    • How many times do users click on the 'call to action' (other than 'OK') (T259775)?
    • How many times do users access the special page or the expanded drawer to see the reason (T259755)?
  • We will use statsd to capture these numbers and we don't log the type of CTA.
Note

This has been done in the past. Related task - T201719: Tracking blocks: Log when the mobile web editor block notice is displayed
Q: Do we have any of the past data still available?

Event Timeline

Niharika triaged this task as Medium priority.Aug 12 2020, 4:51 AM
Niharika created this task.

@jwang I want to flag this task for you. We can talk about it when we meet next.

Niharika removed the point value for this task.
Niharika set the point value for this task to 3.Aug 19 2020, 5:06 PM

As discussed with @Niharika and @jwang:

  • The calls to action are "Log in" and "See random page". There's also "OK", but that dismisses the drawer and is probably less interesting.
  • We need to know (1) how many times was drawer shown, (2) how many times were the interesting calls to action shown, (3) how many times were the interesting calls to action clicked on
ARamirez_WMF changed the subtype of this task from "Task" to "Deadline".Sep 9 2020, 8:45 PM
ARamirez_WMF set Due Date to Sep 22 2020, 4:00 AM.

Change 626710 had a related patch set uploaded (by Tchanders; owner: Tchanders):
[mediawiki/extensions/MobileFrontend@master] Reinstate analytics tracking for the block message drawer

https://gerrit.wikimedia.org/r/626710

Change 626710 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Reinstate analytics tracking for the block message drawer

https://gerrit.wikimedia.org/r/626710

@Tchanders we are not seeing this data in Grafana yet. Jennifer built a dashboard to track it here. The old data is available in the dashboard if you zoom out.
If we are capturing the data the same way, it should show up in the dashboard, correct?

Change 629691 had a related patch set uploaded (by Tchanders; owner: Tchanders):
[operations/mediawiki-config@master] Enable mobile block notice tracking in MobileFrontend

https://gerrit.wikimedia.org/r/629691

@Niharika This is only on 1.36.0-wmf.10, so it wouldn't be giving any significant data yet. It also needs to be enabled via a config change, which we'll deploy later today - thanks for the reminder.

Change 629691 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable mobile block notice tracking in MobileFrontend

https://gerrit.wikimedia.org/r/629691

Mentioned in SAL (#wikimedia-operations) [2020-09-24T19:04:05Z] <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: bcf9fcbe3b82ab85b8f97206ceca45b64619c362: Enable mobile block notice tracking in MobileFrontend (T260218) (duration: 01m 04s)

@Niharika This is only on 1.36.0-wmf.10, so it wouldn't be giving any significant data yet. It also needs to be enabled via a config change, which we'll deploy later today - thanks for the reminder.

Awesome, thanks! @jwang can you double check if the data is showing up now?

I saw the data after 2020-09-24 19:00:00 showing up. Thanks!