Page MenuHomePhabricator

Investigate 5.4.1 related user behavior data changes
Closed, ResolvedPublic1 Estimated Story Points

Description

There seem to be two non-organic changes in page views (increase by a few %) and retention (drop by a few percent) around the time of the 5.4.1 release. Why?

Background activity or thread changes for performance or battery? I know we did some of that... maybe affected the rates here?

Event Timeline

iOS app pageviews (2016-04-01..2017-08-01).png (499×1 px, 72 KB)

(Note: This data already has these corrections for the earlier pageview bug applied, which however didn't completely remove those extraneous views between December and March. Thus that later rise with 5.4.1 may appear as a return to previous levels here, but isn't.)

Also, @JKatzWMF noted that the retention slip began, esp in the Apple iTC data, around the time we had issues with deeplinnking, and that he beleives Apple's deepinking to the app is no longer as effective/aggressive. I've been playing around with it more intetnionally since, and I agree, there are use cases of deeplinks who's behavior seems to have changed to keep you in the browser. That is, it seems that deeplinks from texts are opening in app for me, but Safari searches and links in Safari, now keep you in the browser. This would also explain a drop in unique users over the last 30 days, which also occurred in the late spring, and I originally attributed to an an iTunes analytics issue Apple reported for 10.3.0 users.

I have opened a subtask to look specifically into this theory. Sadly deeplinks has always been the hardest feature to test, due to the opaque and unclearly documented way the OS handles them. I also have opened multiple radar tickets with apple about various deeplink issues, but have never gotten any real traction or insight. Still, I very much worth investigating.

Starting in 5.4.1 the app would no longer report days since install if the app was launched by the OS in the background.