However, 30 of these events report the exact same, extremely long time as their page visible time, implying that the page was never hidden. This is clearly incorrrect.

It's possible that the instrumentation script doesn't auto-detect the screen off when user left the Wikipedia app opens, therefore visible time count.

Another manifestation involves multiple events with the same pageview token that are separated by another event with a different pageview token (non-consecutive pageviews). This could be correct if it reflects a user viewing a page, navigating to another, and then using the back button to return the first page, but in any case, the first page had to have been hidden while the second page was viewed. But, among the 181 non-consecutive KaiOS pageview in our dataset, 88% had the same page visible and page open time. This is also clearly incorrect.

Just search for the instrumentation code, useTracking.js#L129, it seems like the problem of lacking the removeEventListener event for the pagehide event, therefore multiple events with same token are being sent. -- Fix #PR

@hueitan do you plan on investigating the first part (screen off detection)?

However, 30 of these events report the exact same, extremely long time as their page visible time, implying that the page was never hidden. This is clearly incorrrect.

It's possible that the instrumentation script doesn't auto-detect the screen off when user left the Wikipedia app opens, therefore visible time count.

At the moment, I don't find any solution to detect screen off even on the browser. If I am right about this, I'm pretty sure this is an edge case, the possible reason why you see 30 of these events the same is because of the second part of this ticket (lacking of removeEventListener) that we fixed couple of weeks ago.

So I'm asking @nshahquinn-wmf, can you run the query again whether you still see many cases has extremely long visible time which doesn't make sense.

@hueitan On the banana phone, I can confirm that visibilitychange fires correctly when I close the app, close the lid (which turns off the screen), let it close itself after 1 minute of inactivity, or close it with the "power off" button on the side.

Thanks for the finding @SBisson, I do check again on the Nokia flip phone, I can also confirm that visibilitychange detects the screen off (including flip the lid and inactivity screen off).

The second part of this ticket is fixed and pushed to master on 31st March, @nshahquinn-wmf, if you do the query could you do the range starting from April?

@hueitan, thanks for investigating! I discussed this with @AMuigai and @SBisson on Tuesday in the weekly sync, and since it's possible that the old code was runnigng even last week, we decided that I would look at this at the start of next week.