Page MenuHomePhabricator

Metric data for Revision Slider
Closed, InvalidPublic

Description

Context: The revisionSlider going live now, we want to find out how helpful and popular the tool is.
Task: Get data to answer the following questions:

  • How popular is the revisionSlider compared to other beta-features? I.e. what activity do people do when they go on the beta-feature page since the moment the revisionSlider is enabled (not enable anything / enable revisionSlider / enable feature Z /....)
  • From the people who enabled the revisionSlider, how many disable it again and after which time frame?
  • From the people who enabled the revisionSlider, how often do they use it from the time they look at the diff view? (Maybe count clicks on revslider elements per open page?)
  • From the people who enabled the revisionSlider, how often do they use the history link on the diff view page (Can we combine that with the question before?)
  • How much time is spent in the history view/diff view realm per session before another page is visited, differentiated by people with and without revision slider (Can we make that group better comparable by only looking at "hard core users" or so?)
  • How often is the history page being called in general in comparison to before, without revision slider?

Event Timeline

How popular is the revisionSlider compared to other beta-features? I.e. what activity do people do when they go on the beta-feature page since the moment the revisionSlider is enabled (not enable anything / enable revisionSlider / enable feature Z /....)

This seems rather complex, would simply tracking the number of users that have the beta feature enabled suffice?
(This does kind of sound like something that could be rolled out for beta features across the board if it doesn't already exist)

From the people who enabled the revisionSlider, how many disable it again and after which time frame?

This is defiantly possible, it would require recording which users have it enabled (say every day) and comparing that list with the previous day, and recording the number. Basically the number of people that stopped using the beta feature on a given day. Again this sounds like something which may make sense for all beta features.

From the people who enabled the revisionSlider, how often do they use it from the time they look at the diff view? (Maybe count clicks on revslider elements per open page?)

So we already count how many times the RevisonSlider is loaded, a pointer is moved, an arrow (at either end of the timeline) is clicked, and a user goes backwards or forwards in history using the back / forward buttons.
Are there any other elements that you would like recorded?
It would perhaps make sense to count how many times an api call is needed in order to load more edits as a user goes further and further back.
Also perhaps how many times the help dialogue is accessed / instantly dismissed?

From the people who enabled the revisionSlider, how often do they use the history link on the diff view page (Can we combine that with the question before?)

So when the RevisionSlider is loaded and the user is on the diff page and the user then clicks the history tab at the top?
May be able to do this with some javascript.

How much time is spent in the history view/diff view realm per session before another page is visited, differentiated by people with and without revision slider (Can we make that group better comparable by only looking at "hard core users" or so?)

This sounds hard, without even thinking about a 'better group' of users. Again probably possible with some nasty JS but not sure we want to go down that route, especially as this would require the RevisonSlider adding JS even when it isn't being used.

How often is the history page being called in general in comparison to before, without revision slider?

The history page or diff pages? This is very possible, and will possibly look something like T138500.

Can we find out if there is some basic tracking for beta features available already? Because the numbers would be great! And knowing how many people looking at the beta feature page actually enabled the revisionSlider would give us a much better insight about traction than just knowing how many people have it enabled in general.

Are there any other elements that you would like recorded?

I think the elements are sufficient, but would it be possible to record them session wise? So we know if people who had the chance to use the rev slider actually used it and to what extent?

How much time is spent in the history view/diff view realm per session before another page is visited, differentiated by people with and without revision slider (Can we make that group better comparable by only looking at "hard core users" or so?)

Then I guess we should drop this

The history page or diff pages?

The history page. Hypothesis is that the revision slider dramatically reduces the need to use the history page, since you navigate from within the diff view. Although now that you mention it, it might also be interesting if we can see an increase in diff page views :)

Can we find out if there is some basic tracking for beta features available already? Because the numbers would be great! And knowing how many people looking at the beta feature page actually enabled the revisionSlider would give us a much better insight about traction than just knowing how many people have it enabled in general.

I had a quick search around and couldn't find anything.
If we are doing it for revisionslider also doing it for everything else wouldn't be that hard / take any more time.
It may make sense to finish tackling T125989 before doing this though!

I think the elements are sufficient, but would it be possible to record them session wise? So we know if people who had the chance to use the rev slider actually used it and to what extent?

It would be possible to do more advanced tracking through EventLogging!

The history page. Hypothesis is that the revision slider dramatically reduces the need to use the history page, since you navigate from within the diff view. Although now that you mention it, it might also be interesting if we can see an increase in diff page views :)

Hmm, okay, defiantly possible!

@Lea_WMDE this should probably be split into individual tasks.

@Addshore I just opened another window to do that - close to telepathy :)