Page MenuHomePhabricator

InukaPageView web has very little data from Android
Open, Needs TriagePublic

Description

Early results indicate that we receive a lot of data from iOS devices, a small but maybe reasonable amount from KaiOS, but extremely little from Android.

This task is to track the changes to the instrumentation code to try to boost the Android numbers.

Event Timeline

SBisson created this task.Mar 17 2020, 6:20 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 17 2020, 6:20 PM

Change 580406 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/WikimediaEvents@master] Tweak InukaPageView tracking to try to get more Android data

https://gerrit.wikimedia.org/r/580406

@nshahquinn-wmf please have a look at the patch above.

Change 580406 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Tweak InukaPageView tracking to try to get more Android data

https://gerrit.wikimedia.org/r/580406

SBisson moved this task from Code Review to QA on the Inuka-Team (Kanban) board.Mar 19 2020, 12:56 PM
Jpita added a subscriber: Jpita.Mar 21 2020, 2:58 AM

@SBisson can I move this to done or something needs to be checked here?

@SBisson can I move this to done or something needs to be checked here?

I would rather keep this around until it is deployed (next Thursday, hopefully) and we can confirm that it helps get more Android data.

@nshahquinn-wmf this is now in production.

@nshahquinn-wmf this is now in production.

Well, we are now getting about 10 times as much data from KaiOS web clients! However, there was no change in the amount of Android data.

dateandroid-webios-webkaios-appkaios-web
2020-03-21187192330665510025
2020-03-22373213710243812566
2020-03-231701799231116611917
2020-03-241321791527129412057
2020-03-25329193064718512077
2020-03-26 [change deployed]7042022451205218527
2020-03-273922191033774164892
2020-03-2834217601490148701

The KaiOS increase is weird: it's probably due to the regex change, but if so, why did simply being classified by our regex as Android keep them from being sampled?

The KaiOS increase is weird: it's probably due to the regex change, but if so, why did simply being classified by our regex as Android keep them from being sampled?

KaiOS sampling rate is 100% but Android is 10% (was initially 4%). So 90-96% of those KaiOS users classified as Android were excluded.

nshahquinn-wmf added a comment.EditedMar 30 2020, 6:02 PM

The KaiOS increase is weird: it's probably due to the regex change, but if so, why did simply being classified by our regex as Android keep them from being sampled?

KaiOS sampling rate is 100% but Android is 10% (was initially 4%). So 90-96% of those KaiOS users classified as Android were excluded.

Definitely, but our daily number of KaiOS web events went up by about 140,000. 4% of that is about 5,500, and Android web definitely didn't go down by that amount. So, my point stands: only our classification of the devices changed, not anything about the devices themselves, and yet that made us start receiving data from them.

Rileych removed SBisson as the assignee of this task.Mar 31 2020, 5:41 PM
AMuigai moved this task from Kanban to Analyst on the Inuka-Team board.Apr 6 2020, 8:54 PM
AMuigai edited projects, added Inuka-Team; removed Inuka-Team (Kanban).
nshahquinn-wmf moved this task from Analyst to Backlog on the Inuka-Team board.Apr 13 2020, 4:30 PM
nshahquinn-wmf added a subscriber: Rileych.

@AMuigai, @Rileych FYI, this is primarily within the engineers' sphere since the issue is almost certainly with the instrumentation code. Of course, I'm still here to help with brainstorming or checking hypothesis against the existing data!