Page MenuHomePhabricator

Proposal for excluding desktop downloads from the iOS app download number we are reporting
Open, LowPublic

Description

iOS downloads is one of our core metrics which has been tracked on a regular basis since around 2015 and reported in the Readers team's quarterly metrics meetings and in the Audiences department's monthly readers metrics update.

We saw some artifacts on Nov 13 and 29 2017 from App Annie’s download number, and some spikes from their promotion number in other days. In fact, the units reported by iTunes Connect "Sales and Trends" (when the time zone is set to PST) is equal to the sum of App Annie's downloads and promotions under the "Downloads" tab (see iOS analytics document for more details). And on iTunes Connect, if we breakdown the number by device, we could see those unusual bumps all came from desktop.

Based on our email exchange with App Annie and this discussion on Apple Developer Forums, these downloads from desktop are most likely from Volume Purchase Program, either for educational institutions, or for large enterprises. The way these institutions distribute apps to staff internally is via some form of Mobile Device Management system. There are apps they have to distribute to staff/students, and these apps are purchased in a way that shows up as a desktop purchase in the stats. This makes sense as the person purchasing the apps is most likely sat at a desktop computer putting the order through the system. Unlike a user purchasing and downloading the app, this is like a 'right to download' this many copies of the app, and not actual downloads. See this note for more details.

Although Volume Purchase Program can't explain all the spikes, downloading app on desktop is not a usual behavior, especially since version 12.7 Apple removes the desktop iOS app store. See this announcement and this news article.

Therefore, I suggest we exclude desktop downloads from the iOS app download number we are reporting.

Event Timeline

chelsyx created this task.May 11 2018, 11:29 PM
chelsyx triaged this task as Low priority.
chelsyx moved this task from Triage to Doing on the Product-Analytics board.

Thanks (belatedly) for solving this mystery by following up with App Annie and doing further research on this! BTW this is also consistent with the earlier observation that these spikes have always been confined to a single country and a single day, and appear to increase the baseline by a round number like 10,000.

I guess though that there are still arguments both for and against excluding them. If an educational institution chooses to prepare the Wikipedia app for installation on thousands of students' devices for educational purposes, isn't that an important indicator of impact (even if the students still have to complete the actual downloads themselves and not all of them may do so eventually)? I would also conjecture that these organizations don't normally obtain that "right to download" for many more copies than they actually end up using, considering that there is normally a cost involved. (Not for Wikipedia of course, but then again they probably wouldn't need to use these volume purchases for our app in the first place.) Lastly, for those student downloads or installs that are actually occurring as result of a volume purchase, it seems that they won't show up separately in our download metrics, correct?

Or in other words: Maybe Apple's choice to include these in the reported download numbers actually makes sense and is fine for us to adapt - industry standard and all. In any case, my earlier guess that these spikes are artifacts (in the sense of being spurious and not corresponding to any human action) was likely wrong in the light of your findings.

Vvjjkkii renamed this task from Proposal for excluding desktop downloads from the iOS app download number we are reporting to 02caaaaaaa.Jul 1 2018, 1:10 AM
Vvjjkkii raised the priority of this task from Low to High.
Vvjjkkii removed chelsyx as the assignee of this task.
Vvjjkkii updated the task description. (Show Details)
CommunityTechBot assigned this task to chelsyx.
CommunityTechBot lowered the priority of this task from High to Low.
CommunityTechBot renamed this task from 02caaaaaaa to Proposal for excluding desktop downloads from the iOS app download number we are reporting.