Upon recent Android beta release (2/6), EventGate validation errors related to app_version are occurring. Fix the AgentData data object in the library to set the data type of app_version to integer
See https://logstash.wikimedia.org/goto/e1d48f3b75968ff88e09f45c20211fb9
The Android team just released a big feature so they are doing another beta release tomorrow 2/8 to include bug fixes. They are graciously waiting for this update and will include it if I can get an Android MR out in the morning.
cc @phuedx (requesting expedient review), @WDoranWMF (so you're in the loop - I started validating data after the last beta release which went out yesterday 2/6 and found another bug so jumping on it)
phuedx merged https://gitlab.wikimedia.org/repos/data-engineering/metrics-platform/-/merge_requests/29
[Java] Fix validation error for app_version, clean up
New library release:
https://repo1.maven.org/maven2/org/wikimedia/metrics/metrics-platform/2.2/
Android PR:
https://github.com/wikimedia/apps-android-wikipedia/pull/4459
Hopefully ^^ will be merged shortly and included in the next imminent beta release which is scheduled for this morning
Android team did another beta release yesterday and relieved to report that data coming back from v2.2 looks good.
Example queries + results (abbreviated rows):
SELECT * FROM android_product_metrics_article_link_preview_interaction WHERE year=2024 AND month=2 AND day=8;
which returns 19 rows (3 rows shown below):
| _schema | action | action_context | action_source | action_subtype | agent | dt | element_friendly_name | element_id | funnel_entry_token | funnel_event_sequence_position | funnel_name | http | mediawiki | meta | page | performer | sample | user_agent_map | is_wmf_domain | normalized_host | datacenter | year | month | day | hour |
| NULL | article_link_preview_interaction | time_spent_ms.1 | 2 | linkclick | {"app_flavor":"devdebug","app_install_id":"4db76505-039d-4506-8ad6-85ec650a374e","app_theme":"LIGHT","app_version":470,"client_platform":"android","client_platform_family":"app","device_language":"en","release_status":"dev"} | 2024-02-08T22:38:32Z | NULL | NULL | NULL | NULL | NULL | {"has_cookies":null,"method":null,"protocol":null,"request_headers":{"user-agent":"Metrics Platform Client/Java 2.2"},"response_headers":null,"status_code":null} | {"database":"enwiki"} | {"domain":"en.wikipedia.org","dt":"2024-02-08T22:38:53.495Z","id":"0565bb25-29be-4c77-9ffb-e81460e12b56","request_id":"ad9607af-d10d-4ce2-9884-4395295dbb53","stream":"android.product_metrics.article_link_preview_interaction","uri":null} | {"title":"Pablo_Escobar","content_language":"en","id":161570,"namespace_id":0,"namespace_name":null,"revision_id":null,"wikidata_qid":"Q187447"} | {"groups":[],"id":null,"is_bot":null,"is_logged_in":false,"is_temp":null,"language_groups":"[en]","language_primary":"en","name":null,"pageview_id":"abcc6da57fa5b598ea66","registration_dt":null,"session_id":"c8302958b0e64e525caf"} | {"rate":null,"unit":null} | {"browser_family":"Other","browser_major":"-","device_family":"Other","os_family":"Other","os_major":"-","os_minor":"-","wmf_app_version":"-"} | TRUE | {"project_class":"wikipedia","project":"en","qualifiers":[],"tld":"org","project_family":"wikipedia"} | codfw | 2024 | 2 | 8 | 22 |
| NULL | article_link_preview_interaction | time_spent_ms.0 | 2 | linkclick | {"app_flavor":"devdebug","app_install_id":"4db76505-039d-4506-8ad6-85ec650a374e","app_theme":"LIGHT","app_version":470,"client_platform":"android","client_platform_family":"app","device_language":"en","release_status":"dev"} | 2024-02-08T22:38:31Z | NULL | NULL | NULL | NULL | NULL | {"has_cookies":null,"method":null,"protocol":null,"request_headers":{"user-agent":"Metrics Platform Client/Java 2.2"},"response_headers":null,"status_code":null} | {"database":"enwiki"} | {"domain":"en.wikipedia.org","dt":"2024-02-08T22:38:53.495Z","id":"941bf31e-c080-4db0-a2b1-938d503f119c","request_id":"ad9607af-d10d-4ce2-9884-4395295dbb53","stream":"android.product_metrics.article_link_preview_interaction","uri":null} | {"title":"Collaborative_partnership","content_language":"en","id":41285246,"namespace_id":0,"namespace_name":null,"revision_id":null,"wikidata_qid":"Q17007233"} | {"groups":[],"id":null,"is_bot":null,"is_logged_in":false,"is_temp":null,"language_groups":"[en]","language_primary":"en","name":null,"pageview_id":"abcc6da57fa5b598ea66","registration_dt":null,"session_id":"c8302958b0e64e525caf"} | {"rate":null,"unit":null} | {"browser_family":"Other","browser_major":"-","device_family":"Other","os_family":"Other","os_major":"-","os_minor":"-","wmf_app_version":"-"} | TRUE | {"project_class":"wikipedia","project":"en","qualifiers":[],"tld":"org","project_family":"wikipedia"} | codfw | 2024 | 2 | 8 | 22 |
| NULL | article_link_preview_interaction | time_spent_ms.1441 | 2 | navigate | {"app_flavor":"devdebug","app_install_id":"4db76505-039d-4506-8ad6-85ec650a374e","app_theme":"LIGHT","app_version":470,"client_platform":"android","client_platform_family":"app","device_language":"en","release_status":"dev"} | 2024-02-08T22:38:29Z | NULL | NULL | NULL | NULL | NULL | {"has_cookies":null,"method":null,"protocol":null,"request_headers":{"user-agent":"Metrics Platform Client/Java 2.2"},"response_headers":null,"status_code":null} | {"database":"enwiki"} | {"domain":"en.wikipedia.org","dt":"2024-02-08T22:38:53.495Z","id":"2ed3dcf1-622b-4faa-9379-c253898e5cb9","request_id":"ad9607af-d10d-4ce2-9884-4395295dbb53","stream":"android.product_metrics.article_link_preview_interaction","uri":null} | {"title":"Medellín_Cartel","content_language":"en","id":357286,"namespace_id":0,"namespace_name":null,"revision_id":null,"wikidata_qid":"Q1046933"} | {"groups":[],"id":null,"is_bot":null,"is_logged_in":false,"is_temp":null,"language_groups":"[en]","language_primary":"en","name":null,"pageview_id":"dfcb54d471fd182f82f2","registration_dt":null,"session_id":"c8302958b0e64e525caf"} | {"rate":null,"unit":null} | {"browser_family":"Other","browser_major":"-","device_family":"Other","os_family":"Other","os_major":"-","os_minor":"-","wmf_app_version":"-"} | TRUE | {"project_class":"wikipedia","project":"en","qualifiers":[],"tld":"org","project_family":"wikipedia"} | codfw | 2024 | 2 | 8 | 22 |
SELECT * FROM android_product_metrics_article_toolbar_interaction WHERE year=2024 AND month=2 AND day=8;
| _schema | action | action_context | action_source | action_subtype | agent | dt | element_friendly_name | element_id | funnel_entry_token | funnel_event_sequence_position | funnel_name | http | mediawiki | meta | page | performer | sample | user_agent_map | is_wmf_domain | normalized_host | datacenter | year | month | day | hour |
| NULL | article_toolbar_interaction | time_spent_ms.23708 | NULL | back | {"app_flavor":"devdebug","app_install_id":"1005168d-3bf8-40dd-b320-0695e98223cd","app_theme":"LIGHT","app_version":470,"client_platform":"android","client_platform_family":"app","device_language":"ru","release_status":"dev"} | 2024-02-08T19:26:46Z | NULL | NULL | NULL | NULL | NULL | {"has_cookies":null,"method":null,"protocol":null,"request_headers":{"user-agent":"Metrics Platform Client/Java 2.2"},"response_headers":null,"status_code":null} | {"database":"enwiki"} | {"domain":"en.wikipedia.org","dt":"2024-02-08T19:26:44.536Z","id":"0f4ca40a-de35-4feb-9270-9b3e335b490f","request_id":"eaa655fb-5e53-4053-bb7f-967cfbe335bf","stream":"android.product_metrics.article_toolbar_interaction","uri":null} | {"title":"Second_War_of_Scottish_Independence","content_language":"en","id":26694798,"namespace_id":0,"namespace_name":null,"revision_id":null,"wikidata_qid":null} | {"groups":["*","user"],"id":null,"is_bot":null,"is_logged_in":true,"is_temp":null,"language_groups":"[en, ru, de, he]","language_primary":"en","name":null,"pageview_id":"faf0ca1320882ca52fb8","registration_dt":null,"session_id":"a6ca6d045f02edcbbd6f"} | {"rate":null,"unit":null} | {"browser_family":"Other","browser_major":"-","device_family":"Other","os_family":"Other","os_major":"-","os_minor":"-","wmf_app_version":"-"} | TRUE | {"project_class":"wikipedia","project":"en","qualifiers":[],"tld":"org","project_family":"wikipedia"} | eqiad | 2024 | 2 | 8 | 19 |
| NULL | article_toolbar_interaction | time_spent_ms.4821 | NULL | back | {"app_flavor":"devdebug","app_install_id":"1005168d-3bf8-40dd-b320-0695e98223cd","app_theme":"LIGHT","app_version":470,"client_platform":"android","client_platform_family":"app","device_language":"ru","release_status":"dev"} | 2024-02-08T19:26:20Z | NULL | NULL | NULL | NULL | NULL | {"has_cookies":null,"method":null,"protocol":null,"request_headers":{"user-agent":"Metrics Platform Client/Java 2.2"},"response_headers":null,"status_code":null} | {"database":"enwiki"} | {"domain":"en.wikipedia.org","dt":"2024-02-08T19:26:44.536Z","id":"4cdf5028-c2c9-44ba-927a-22a930a80aa9","request_id":"eaa655fb-5e53-4053-bb7f-967cfbe335bf","stream":"android.product_metrics.article_toolbar_interaction","uri":null} | {"title":"Kingdom_of_England","content_language":"en","id":407950,"namespace_id":0,"namespace_name":null,"revision_id":null,"wikidata_qid":null} | {"groups":["*","user"],"id":null,"is_bot":null,"is_logged_in":true,"is_temp":null,"language_groups":"[en, ru, de, he]","language_primary":"en","name":null,"pageview_id":"5712dd889576c687d9a2","registration_dt":null,"session_id":"a6ca6d045f02edcbbd6f"} | {"rate":null,"unit":null} | {"browser_family":"Other","browser_major":"-","device_family":"Other","os_family":"Other","os_major":"-","os_minor":"-","wmf_app_version":"-"} | TRUE | {"project_class":"wikipedia","project":"en","qualifiers":[],"tld":"org","project_family":"wikipedia"} | eqiad | 2024 | 2 | 8 | 19 |
| NULL | article_toolbar_interaction | time_spent_ms.5113 | NULL | watch_article | {"app_flavor":"devdebug","app_install_id":"4db76505-039d-4506-8ad6-85ec650a374e","app_theme":"LIGHT","app_version":470,"client_platform":"android","client_platform_family":"app","device_language":"en","release_status":"dev"} | 2024-02-08T23:19:49Z | NULL | NULL | NULL | NULL | NULL | {"has_cookies":null,"method":null,"protocol":null,"request_headers":{"user-agent":"Metrics Platform Client/Java 2.2"},"response_headers":null,"status_code":null} | {"database":"enwiki"} | {"domain":"en.wikipedia.org","dt":"2024-02-08T23:19:58.879Z","id":"cc77e99e-c605-49c1-839f-6c9076af4d8f","request_id":"a578d9ad-c9fd-4e53-b8cd-1ee74d17f544","stream":"android.product_metrics.article_toolbar_interaction","uri":null} | {"title":"President_of_Chile","content_language":"en","id":11885300,"namespace_id":0,"namespace_name":null,"revision_id":null,"wikidata_qid":null} | {"groups":["*","user"],"id":null,"is_bot":null,"is_logged_in":true,"is_temp":null,"language_groups":"[en]","language_primary":"en","name":null,"pageview_id":"c6e88b33b6737c9825bb","registration_dt":null,"session_id":"c8302958b0e64e525caf"} | {"rate":null,"unit":null} | {"browser_family":"Other","browser_major":"-","device_family":"Other","os_family":"Other","os_major":"-","os_minor":"-","wmf_app_version":"-"} | TRUE | {"project_class":"wikipedia","project":"en","qualifiers":[],"tld":"org","project_family":"wikipedia"} | codfw | 2024 | 2 | 8 | 23 |
SELECT * FROM android_product_metrics_article_toc_interaction WHERE year=2024 AND month=2 AND day=9;
| _schema | action | action_context | action_source | action_subtype | agent | dt | element_friendly_name | element_id | funnel_entry_token | funnel_event_sequence_position | funnel_name | http | mediawiki | meta | num_opens | num_peeks | num_section_clicks | num_sections | page | performer | sample | time_spent_ms | total_open_sec | total_peek_sec | user_agent_map | is_wmf_domain | normalized_host | datacenter | year | month | day | hour |
| NULL | article_toc_interaction | NULL | NULL | NULL | {"app_flavor":"devdebug","app_install_id":"a6b894e0-29f9-4c4b-9edf-58d10c25099b","app_theme":"LIGHT","app_version":470,"client_platform":"android","client_platform_family":"app","device_language":"en","release_status":"dev"} | 2024-02-09T00:39:29Z | NULL | NULL | NULL | NULL | NULL | {"has_cookies":null,"method":null,"protocol":null,"request_headers":{"user-agent":"Metrics Platform Client/Java 2.2"},"response_headers":null,"status_code":null} | {"database":"enwiki"} | {"domain":"en.wikipedia.org","dt":"2024-02-09T00:39:41.239Z","id":"c4e1e0ce-7a3e-4c0e-be32-0050d1087726","request_id":"0432bb55-c529-4c50-bba3-c8180907ac6d","stream":"android.product_metrics.article_toc_interaction","uri":null} | 1 | NULL | 1 | 16 | {"title":"Fibración_de_Hopf","content_language":"es","id":5555356,"namespace_id":0,"namespace_name":null,"revision_id":null,"wikidata_qid":null} | {"groups":[],"id":null,"is_logged_in":false,"is_temp":null,"language_groups":"[en]","language_primary":"en","name":null,"pageview_id":"93390c2c2caaf762e73f","registration_dt":null,"session_id":"06af368cfd50212ec486"} | {"rate":null,"unit":null} | NULL | 1 | NULL | {"browser_family":"Other","browser_major":"-","device_family":"Other","os_family":"Other","os_major":"-","os_minor":"-","wmf_app_version":"-"} | TRUE | {"project_class":"wikipedia","project":"en","qualifiers":[],"tld":"org","project_family":"wikipedia"} | codfw | 2024 | 2 | 9 | 0 |
The last query:
SELECT * FROM android_product_metrics_find_in_page_interaction WHERE year=2024 AND month=2 AND day=9;
^^ isn't returning data yet -- I'm not entirely sure why so I'll keep monitoring and hopefully we'll see data flowing in for this stream soon.
Next step is to let Android data analyst know that reviewing data checks can be resumed.