Spike Questions
- Are the numbers we have from last year over-inflated?
- Are the queries querying the wrong thing?
- Were the events emitted in a different way?
- Are the numbers have from this year under-inflated?
- Are the queries wrong?
- Are the events emitted in a different way?
- Are they being emitted correctly?
Goal
- Is it possible to have confidence in the parity of both the events emitted and our ability to query them?
Background from Shay
Event counts we are questioning:
active_interface = article_banner
action = impression
action_data: campaign_id
platform = ios
in schema app_donor_experience
It looks as if some of these events are firing and tracking successfully, but not all. Banner impression events need to be triggered for all users, all appearances. See: Apps Donor Experience Instrumentation Documentation
When looking at event counts for impression events we saw a significant reduction in events when compared to EN campaign impressions from last year's data. We had implemented measuring impression events in app_donor_experience to replace the deprecated MobileWikiAppiOSFeed. Because of changes to announcement we want to verify that data is being sent properly, though this may also very likely be an issue with user data sharing/increased Apple Privacy restrictions.
I looked at DAUs and events recorded in other schemas (app_session, ios_navigation_events) and saw similarly low event/unique users counts. Additionally looked at user app version distribution for app_donor_experience EN impression events (result was 75.4% version 7.4.5.2871 and 24.4% version 7.4.6.2972). Investigative data
iOS Banner impression counts used to come from MobileWikiAppiOSFeed, which is deprecated. The query used by us and FR Tech to get impression counts when MobileWikiAppiOSFeed schema was still active was as follows:
SELECT year, month, day, COUNT(*) AS event_count FROM event.MobileWikiAppiOSFeed WHERE year = 2022 AND ( ( month = 11 AND day = 29 ) OR ( month = 11 AND day = 30 ) OR month = 12 ) AND event.action = 'impression' AND ( event.label = 'announcement' OR event.label = 'article_announcement' )
- On my end I will be investigating app uniques using web request image loading events to get an approximate distribution of app DAU uniques by opt-in/opt-out based on uniques in 'wmfuuid' in x_analytics_map web request data. Thsi will give us an idea of scale for uniques we may be missing with app_session daus. This data will also be helpful to have for when we remove sharing opt-in/opt-out to see effect of change for our data accessibility.
