Page MenuHomePhabricator

The number of unique IDs are very different in MobileWikiAppDailyStats and mobile_apps_uniques
Closed, ResolvedPublic

Description

On any given day, if an opted-in user use the app at least once, the app will send an event to MobileWikiAppDailyStats with an appInstallID generated by WMFDailyStatsLoggingFunnel. MobileWikiAppDailyStats is 100% sampling.
Also, the app will send at least one API call to fetch the feed/article with an wmfuuid in X-Analytics header generated by ReadingActionFunnel (through [L15 in AFHTTPRequestSerializer+WMFRequestHeaders.m](https://github.com/wikimedia/wikipedia-ios/blob/1d2548e869086a279b90c2302feeb4cbadd31b59/Wikipedia/Code/AFHTTPRequestSerializer%2BWMFRequestHeaders.m#L15)). For example, when I tap to read an article "Mount Potalaka" on the app, our server will receive a request like:
en.wikipedia.org/w/api.php?action=mobileview&format=json&noheadings=true&page=Mount%20Potalaka&pilicense=any&prop=sections%7Ctext%7Clastmodified%7Clastmodifiedby%7Clanguagecount%7Cid%7Cprotection%7Ceditable%7Cdisplaytitle%7Cthumb%7Cdescription%7Cimage%7Crevision%7Cnamespace&sectionprop=toclevel%7Cline%7Canchor%7Clevel%7Cnumber%7Cfromtitle%7Cindex&sections=all&thumbwidth=640
Along with this API call, the X-Analytics header which contains an wmfuuid will be sent, like:
ns=0;page_id=25807626;loggedIn=1;WMF-Last-Access=05-Mar-2018;WMF-Last-Access-Global=05-Mar-2018;https=1;wmfuuid=3A5FC461-5F94-4B85-9857-64A552DF7D98
(See T189359#4179759 for steps to reproduce)
The API call, X-Analytics header and other information are stored in the [webrequest](https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/Webrequest) table in our hadoop cluster. We count the number of unique wmfuuid in webrequest table to get the number of unique devices, which is stored in the [mobile_apps_uniques](https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/mobile_apps_uniques) table (see the query).

Although these two IDs are different, we should have similar (if not the same) counts of unique IDs in MobileWikiAppDailyStats and [mobile_apps_uniques](https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/mobile_apps_uniques) tables. However, we have only ~65k distinct iOS appInstallIDs daily in MobileWikiAppDailyStats, while there are ~140k distinct iOS appInstallIDs daily in mobile_apps_uniques. See the following queries:

-- Get the daily number of unique IDs from mobile_apps_uniques
SELECT *
FROM wmf.mobile_apps_uniques_daily
WHERE platform = 'iOS'
AND year=2018 AND month=4
-- Get the daily number of unique IDs from MobileWikiAppDailyStats
SELECT year, month, day, COUNT(DISTINCT event.appinstallid)
FROM event.mobilewikiappdailystats
WHERE useragent.os_family = 'iOS'
AND year=2018 AND month=4
GROUP BY year, month, day
ORDER BY day
LIMIT 30

This issue may be related to T189359 and T189356. We will check again after these two bugs are fixed.

Event Timeline

Note to myself: Check the number of unique IDs by app version.

Thanks to the fix of T189356, this investigation now becomes easier :)

I compared the unique appInstallIDs in MobileWikiAppDailyStats vs mobile_apps_uniques, and found that the different IDs are from the following types of API calls:

api/rest_v1/metrics/pageviews/
api/rest_v1/feed/featured/
api/rest_v1/feed/onthisday/events/

However, I can't figure out how to print the X-Analytics header when these API calls are triggered in the app to verify. @NHarateh_WMF Can you help? Thanks!

@chelsyx

If you set up a breakpoint on this line https://github.com/wikimedia/wikipedia-ios/blob/1d2548e869086a279b90c2302feeb4cbadd31b59/Wikipedia/Code/WMFBaseRequestSerializer.m#L17 and then right click the breakpoint, choose Edit Breakpoint... and add this as the condition:

[URLString containsString:@"rest_v1/metrics/pageviews/"] || [URLString containsString:@"rest_v1/feed/featured/"] || [URLString containsString:@"rest_v1/feed/onthisday/events"]

it should break if any of the calls you identified are about to be performed, in the console you can po mutableRequest.allHTTPHeaderFields and see the appInstallID

After investigation, we believe the difference didn't come from a bug -- we didn't generate multiple appInstallIDs since v5.8.1. Instead, we're seeing more unique IDs in mobile_apps_uniques (aggregation of the webrequest table) than in MobileWikiAppDailyStats because:

1, We started to send events to MobileWikiAppDailyStats since v5.0.0 . Apps of older version (before v5.0.0) would only be counted in mobile_apps_uniques, not MobileWikiAppDailyStats.
2, The app sends event to MobileWikiAppDailyStats only when it is in the foreground. When the app is in the background, it may fetch feed and reading lists updates when the system allows it (we don’t control when that happens). Therefore, even if the app is not active on certain days (users didn't bring it to the foreground and thus no event was sent to MobileWikiAppDailyStats), we may still see the app's ID in webrequest table.

Vvjjkkii renamed this task from The number of unique IDs are very different in MobileWikiAppDailyStats and mobile_apps_uniques to qkdaaaaaaa.Jul 1 2018, 1:11 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii removed chelsyx as the assignee of this task.
Vvjjkkii raised the priority of this task from Medium to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
CommunityTechBot renamed this task from qkdaaaaaaa to The number of unique IDs are very different in MobileWikiAppDailyStats and mobile_apps_uniques.Jul 2 2018, 2:01 PM
CommunityTechBot closed this task as Resolved.
CommunityTechBot assigned this task to chelsyx.
CommunityTechBot lowered the priority of this task from High to Medium.
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot added a subscriber: Aklapper.