Page MenuHomePhabricator

Homepage: analyze hovers
Open, Needs TriagePublic

Description

In T216586, we implemented code to record hover-in and hover-out events for desktop users of the homepage, indicating how long they focus their mouse on each module. According to our measurement plan, there are some specific questions we want to answer with these hover events, but on doing this task, we should double check the measurement plan to make sure we got them all.

  • How much time do users spend interacting with homepage modules per visit?
    • During their first visit?
    • During subsequent visits?
  • How long do desktop users hover over each module? (analogous measurement for mobile depends on pending mobile design)

Related Objects

Event Timeline

RHo added a comment.Aug 7 2019, 9:22 PM

Some additional questions:

  • How do hovers compare with click interactions in each module? (Are there some modules where users hovered in for a long time but have low click rates?)
  • Do we have data on the average/median hover duration before clicking in various modules? Vs hover out?

I'm assuming significant disparity between hovers and clicks indicates that the user is initially interested in the module content but decided not to click it because it didn't contain the action they wanted. This may help inform design improvements to the various modules (for example, help answer to what extent the "Impact" module has lower interactions than the "Start" module due to its placement vs its content).

However, IANADA, so @nettrom_WMF please feel free to correct me about how to interpret this data,

kostajh moved this task from Inbox to Q1 2019-20 on the Growth-Team board.Aug 19 2019, 12:11 PM

Seems like something for us to sort out in Q1 but @MMiller_WMF feel free to move.

I went back and looked through the measurement plan in order to see what we proposed there and what discussions we had around these measurements. The three questions in the task description appear to be all of them. One thing that isn't captured above is the note that the first question is our proxy for "time on page", which we use in placement of a ping-based approach to learn how long users stay on the page. The concern was that users left the Homepage too quickly, as reflected in our leading indicators (where we found that they don't).

Another thing that isn't mentioned is the discussion around using hovers for measuring time on page in the measurement plan. One comment in that discussion mentions that we should primarily rely on click-through data, and have the hovers as a backup. So my first question about this analysis is whether we're seeing something in the click data that we're reporting that suggests that we're concerned about how specific modules are performing?

If we are concerned, what modules are we concerned about, and how will the design of the Homepage change depending on what the findings of this analysis are?

2019 abu. 28 7:34 PM erabiltzaileak hau idatzi du (nettrom_WMF <no-reply@phabricator.wikimedia.org>):
nettrom_WMF added a comment.
View Taskhttps://phabricator.wikimedia.org/T230073

I went back and looked through the measurement plan in order to see what we proposed there and what discussions we had around these measurements. The three questions in the task description appear to be all of them. One thing that isn't captured above is the note that the first question is our proxy for "time on page", which we use in placement of a ping-based approach to learn how long users stay on the page. The concern was that users left the Homepage too quickly, as reflected in our leading indicators (where we found that they don't).

Another thing that isn't mentioned is the discussion around using hovers for measuring time on page in the measurement plan. One comment in that discussion mentions that we should primarily rely on click-through data, and have the hovers as a backup. So my first question about this analysis is whether we're seeing something in the click data that we're reporting that suggests that we're concerned about how specific modules are performing?

If we are concerned, what modules are we concerned about, and how will the design of the Homepage change depending on what the findings of this analysis are?

TASK DETAIL
https://phabricator.wikimedia.org/T230073

EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: nettrom_WMF
Cc: Aklapper, JTannerWMF, RHo, Trizek-WMF, nettrom_WMF, Catrope, kostajh, SBisson, Etonkovidova, kaldari, marcella, DannyH, ppelberg, Cntlsn, leila, Theklan, MMiller_WMF, Urbanecm, Taiwania_Justo

@nettrom_WMF and I have decided that this is a lower priority than other analysis tasks we need to do during Q1, and so I am moving it to Q2.