Page MenuHomePhabricator

Visible time for KaiOS pageviews is not correctly counted
Closed, ResolvedPublic

Description

In some cases, InukaPageView data sent from the KaiOS app has incorrect page visible times.

One manifestation of this is extremely long times. In the past week's development data, there were 34 events with page open times greater than 3 days (all from KaiOS devices, so we know this is likely to affect real users). Theoretically at least, this could be correct if a user closes the app, and then reopens it later to show the same page days later.

However, 30 of these events report the exact same, extremely long time as their page visible time, implying that the page was never hidden. This is clearly incorrrect.

Another manifestation involves multiple events with the same pageview token that are separated by another event with a different pageview token (non-consecutive pageviews). This could be correct if it reflects a user viewing a page, navigating to another, and then using the back button to return the first page, but in any case, the first page had to have been hidden while the second page was viewed. But, among the 181 non-consecutive KaiOS pageview in our dataset, 88% had the same page visible and page open time. This is also clearly incorrect.

Event Timeline

hueitan added a comment.EditedMar 29 2020, 3:02 PM

However, 30 of these events report the exact same, extremely long time as their page visible time, implying that the page was never hidden. This is clearly incorrrect.

It's possible that the instrumentation script doesn't auto-detect the screen off when user left the Wikipedia app opens, therefore visible time count.

Another manifestation involves multiple events with the same pageview token that are separated by another event with a different pageview token (non-consecutive pageviews). This could be correct if it reflects a user viewing a page, navigating to another, and then using the back button to return the first page, but in any case, the first page had to have been hidden while the second page was viewed. But, among the 181 non-consecutive KaiOS pageview in our dataset, 88% had the same page visible and page open time. This is also clearly incorrect.

Just search for the instrumentation code, useTracking.js#L129, it seems like the problem of lacking the removeEventListener event for the pagehide event, therefore multiple events with same token are being sent. -- Fix #PR

@hueitan do you plan on investigating the first part (screen off detection)?

AMuigai triaged this task as Medium priority.Apr 4 2020, 6:06 AM
hueitan claimed this task.Apr 8 2020, 12:36 PM
hueitan moved this task from Ready for Dev to Dev on the Inuka-Team (Kanban) board.

Sure, since second part is fixed, let me continue the first part.

However, 30 of these events report the exact same, extremely long time as their page visible time, implying that the page was never hidden. This is clearly incorrrect.

It's possible that the instrumentation script doesn't auto-detect the screen off when user left the Wikipedia app opens, therefore visible time count.

At the moment, I don't find any solution to detect screen off even on the browser. If I am right about this, I'm pretty sure this is an edge case, the possible reason why you see 30 of these events the same is because of the second part of this ticket (lacking of removeEventListener) that we fixed couple of weeks ago.

So I'm asking @nshahquinn-wmf, can you run the query again whether you still see many cases has extremely long visible time which doesn't make sense.

@hueitan On the banana phone, I can confirm that visibilitychange fires correctly when I close the app, close the lid (which turns off the screen), let it close itself after 1 minute of inactivity, or close it with the "power off" button on the side.

Thanks for the finding @SBisson, I do check again on the Nokia flip phone, I can also confirm that visibilitychange detects the screen off (including flip the lid and inactivity screen off).

The second part of this ticket is fixed and pushed to master on 31st March, @nshahquinn-wmf, if you do the query could you do the range starting from April?

@hueitan, thanks for investigating! I discussed this with @AMuigai and @SBisson on Tuesday in the weekly sync, and since it's possible that the old code was runnigng even last week, we decided that I would look at this at the start of next week.

hueitan removed hueitan as the assignee of this task.Apr 27 2020, 6:52 AM
nshahquinn-wmf closed this task as Resolved.May 6 2020, 7:34 PM
nshahquinn-wmf claimed this task.

We haven't seen any page visible times longer than an hour in the past few weeks, so it seems like this has been fixed!

nshahquinn-wmf removed nshahquinn-wmf as the assignee of this task.May 6 2020, 7:34 PM