Page MenuHomePhabricator

Growth team requests for Schema:VisualEditorFeatureUse
Closed, ResolvedPublic

Description

In a call today that included @Neil_P._Quinn_WMF, @DLynch, @marcella, @JTannerWMF, @nettrom_WMF, @Catrope, and @MMiller_WMF, we discussed a few things that it would be possible to include in the new VisualEditorFeatureUse schema that could be really useful to Growth team's Understanding first day project.

Growth team hopes that the following things can be instrumented:

  • Question mark icon: we would like to track when users click on the "?" icon in the toolbar of the editor, and when they select any of the three options that come up ("Read the user guide", "Keyboard shortcuts", "Leave feedback about this software")

  • Blue helper dots: we would like to track user interaction with either of the two blue pulsing helper dots on the "Cite" and "Link" tools in the toolbar. Those are present when a user is in the editor for the first time, until they click either "Cite" or "Link". Therefore, we essentially just want to make sure we are tracking when users click "Cite" and "Link".

  • Notices: we would like to track when users click on the "Notices" icon.

Event Timeline

MMiller_WMF triaged this task as High priority.Oct 23 2018, 10:19 PM
MMiller_WMF created this task.

For the dots, is knowing when the cite/link feature is used sufficient, or do you need specifically that it was a toolbar interaction? (Or even more specifically, a toolbar interaction while the dot is present?) If it's the former, we do have that already.

Further question: the notices popup opens by default if there are any notices. Is what you want effectively whether people re-open it?

Change 469631 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/VisualEditor@master] Analytics: track more things for Growth

https://gerrit.wikimedia.org/r/469631

That patch tracks the blue dots and the ? button. It's explicitly ignoring notices while I await an answer on the initially-open thing.

Thanks for working on this, @DLynch, and for the clarification questions.

For blue dots: the thing we're trying to get at is whether users see the blue dots and are like, "Oooh, I wonder what this is" and go click on it soon after opening the editor for the first time. What do you think will record data that will help us know that? Perhaps all we need is just to know that they use Link or Cite, and if they do it at the beginning of the session, it probably means that they were like "Oooh, I wonder what this is"?

For notices: I forgot that this defaults to open. Yes, we definitely want to know if people re-open it. I think that's all we can really do, because it doesn't really make sense to track closing it. For people for whom it defaults to open, they will close it by doing literally anything besides clicking on it. Maybe the simplest thing is to just track every time someone clicks on its toolbar icon.

mpopov moved this task from Triage to Tracking on the Product-Analytics board.Oct 26 2018, 6:39 PM

Okay, patch is updated to handle these things. Tracking will cover popups opening via an actual click on the button rather than any arbitrary opening. Blue dots will be tracked via the education popup they trigger opening -- see the linkEducationPopup action in the screenshot below, followed by link which means the popup closed and the regular link action continued.

MMiller_WMF closed this task as Resolved.Oct 30 2018, 12:42 AM

Thanks, @DLynch. We're looking forward to seeing the data!

Deskana added a subscriber: matmarex.

Looks like this is still waiting for code review.

@Esanders @matmarex Do either of you think you could get this reviewed before the branch is cut today?

Change 469631 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Analytics: track more things for Growth

https://gerrit.wikimedia.org/r/469631

According to the release notes, this did get merged before the branch was cut, so will be live everywhere today.