Page MenuHomePhabricator

Android investigate DAU counts on `app_session` post auto data sharing in latest app version
Closed, ResolvedPublic

Description

Tracking unique active users by app_install_id day over day by app version we are only seeing a 23.8% increase in new version (2.7.50472-r-2024-02-22) users compared to prior version (2.7.50467-r-2024-01-22) users which allowed for Data Sharing opt-out. I'm keeping an eye on this but want to request that engineering take a look at data sent to app_session to make sure we are polling all latest-version active users. The PlayStore reports an average ~916K DAU, and we are only showing ~133K.

Android Daily DAU Compare by Version - 'app_session'

Note: I ran the same DAU query on stream event.android_article_link_preview_interaction and see that DAU for both versions is much higher than daily than app_session which seems unexpected - if a user is looking at links they are also initiating a session (as I understand).

android_article_link_preview_interaction 3 Day Average DAU:

VersionAvg. DAU% Change
2.7.50467-r-2024-01-22474,813
2.7.50472-r-2024-02-22688,67645.0%

Android Daily DAU Compare by Version - 'event.android_article_link_preview_interaction'

Ran same query for event.android_breadcrumbs_event, DAUs are higher than android_article_link_preview_interaction (which seems ok, these users may only be looking at explore, etc).
3 Day Average DAU:

VersionAvg. DAU% Change
2.7.50467-r-2024-01-22566,209
2.7.50472-r-2024-02-22736,57030.1%

Android Daily DAU Compare by Version - 'event.android_breadcrumbs_event'

On my end I am looking into DAU data we get from PlayStore (see Dashboard) - it's possible users are opting out on devices or (unlikely) PlayStore data is wrong.

For comparison see iOS DAUs https://phabricator.wikimedia.org/T356804 which show a 3851.3% change, closer to what we expected to see with Android.

Event Timeline

Hi @SNowick_WMF

Could you please help to check the data set for a fix that we accidentally set the opt-in default to ture before, even if the user opted out of the data-sharing in the onboarding screen, it will be set as true for data-sharing.
https://github.com/wikimedia/apps-android-wikipedia/pull/4240

Could you please check the versions before and after Oct 20, 2023 to see the difference so I can investigate further?

Before fix
2.7.50455-r-2023-10-10

After fix
2.7.50456-r-2023-11-01

Hi @SNowick_WMF

By checking the AppSessionEvent in the app, it looks like we have a logic to log the app_session only if it has timed out (the default is 30 minutes)
https://github.com/wikimedia/apps-android-wikipedia/blob/302bd3e94dd116b8658f70cb2a781dda7715acf2/app/src/main/java/org/wikipedia/analytics/eventplatform/AppSessionEvent.kt#L84-L87

Since the AppSessionEvent logging actions existed in the app for a long time, I am unsure whether we should remove the logic from a 30 minutes timed out. cc @Dbrant

Thank you @cooltey. Resolving this task, sounds like schema/data is as expected so let's leave logic as-is. Will follow up here with historical info on why we are measuring this way but we don't need to change this.