Page MenuHomePhabricator

InukaPageView web has very little data from Android
Closed, DeclinedPublic

Description

Early results indicate that we receive a lot of data from iOS devices, a small but maybe reasonable amount from KaiOS, but extremely little from Android.

This task is to track the changes to the instrumentation code to try to boost the Android numbers.

Event Timeline

Change 580406 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/WikimediaEvents@master] Tweak InukaPageView tracking to try to get more Android data

https://gerrit.wikimedia.org/r/580406

Change 580406 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Tweak InukaPageView tracking to try to get more Android data

https://gerrit.wikimedia.org/r/580406

@SBisson can I move this to done or something needs to be checked here?

@SBisson can I move this to done or something needs to be checked here?

I would rather keep this around until it is deployed (next Thursday, hopefully) and we can confirm that it helps get more Android data.

@nshahquinn-wmf this is now in production.

Well, we are now getting about 10 times as much data from KaiOS web clients! However, there was no change in the amount of Android data.

dateandroid-webios-webkaios-appkaios-web
2020-03-21187192330665510025
2020-03-22373213710243812566
2020-03-231701799231116611917
2020-03-241321791527129412057
2020-03-25329193064718512077
2020-03-26 [change deployed]7042022451205218527
2020-03-273922191033774164892
2020-03-2834217601490148701

The KaiOS increase is weird: it's probably due to the regex change, but if so, why did simply being classified by our regex as Android keep them from being sampled?

The KaiOS increase is weird: it's probably due to the regex change, but if so, why did simply being classified by our regex as Android keep them from being sampled?

KaiOS sampling rate is 100% but Android is 10% (was initially 4%). So 90-96% of those KaiOS users classified as Android were excluded.

The KaiOS increase is weird: it's probably due to the regex change, but if so, why did simply being classified by our regex as Android keep them from being sampled?

KaiOS sampling rate is 100% but Android is 10% (was initially 4%). So 90-96% of those KaiOS users classified as Android were excluded.

Definitely, but our daily number of KaiOS web events went up by about 140,000. 4% of that is about 5,500, and Android web definitely didn't go down by that amount. So, my point stands: only our classification of the devices changed, not anything about the devices themselves, and yet that made us start receiving data from them.

AMuigai edited projects, added Inuka-Team; removed Inuka-Team (Kanban).
nshahquinn-wmf added a subscriber: Rileych.

@AMuigai, @Rileych FYI, this is primarily within the engineers' sphere since the issue is almost certainly with the instrumentation code. Of course, I'm still here to help with brainstorming or checking hypothesis against the existing data!

It's not clear that this is still worth doing (it's on the agenda for a meeting between Angie, Stephane, and I), but if we do pick it back up, I wanted to note that Kate suggested a while back that Dmitri on the Mobile Apps team could be a good person to ask for ideas about what could be causing this.

Won't be investigated further since we're not collecting InukaPageView from web clients anymore.