Page MenuHomePhabricator

Investigate the Sep 7 drop and the Oct 25 spike in iOS app's 7-day retention
Closed, ResolvedPublicNov 22 2018

Description

We saw a drop since September 7 and a spike on October 25 in the 7-day retention of Wikipedia iOS app users. We will start the investigation from the client side, since these changes happened within 7 days before a new release.

An example of how the 7-day retention is being computed: the 7-day retention of Sep 7 is the proportion of users who install the app on Sep 7 and open the app at least once on Sep 14.

The first step is to count the number of events for those suspicious date.

Details

Due Date
Nov 22 2018, 1:00 AM

Event Timeline

For reference, the implementation task for the underlying instrumentation: T126693

chelsyx triaged this task as High priority.Nov 15 2018, 11:02 PM
chelsyx moved this task from Triage to Doing on the Product-Analytics board.
chelsyx set Due Date to Nov 22 2018, 1:00 AM.
Restricted Application changed the subtype of this task from "Task" to "Deadline". · View Herald TranscriptNov 15 2018, 11:02 PM
chelsyx updated the task description. (Show Details)Nov 15 2018, 11:12 PM

@kzimmerman Now I'm sure the drop and spike are the results of some bug on the app side:

I will QA the app and find out how the bug behaves, then file a ticket to the iOS team.

chelsyx closed this task as Resolved.EditedNov 21 2018, 5:10 AM

In v6.0.1 and v6.0.2, some events are saved in the app but didn't get sent out (see T205551). After upgrading to v6.1, those saved events are sent all at once, that's why we saw a spike after v6.1 released.

I tested and confirmed that this bug no longer exist in the latest version v6.1.2.

I re-computed the 7-day retention with the client side timestamp instead of the system timestamp:


We're not seeing the dip and the spike anymore.