Page MenuHomePhabricator

[Java] Investigate why user_agent_map is empty
Closed, ResolvedPublic2 Estimated Story Points

Description

During resolution of T356938 and initial PA review thereafter, it appears that user_agent_map is sending empty values.

Description

Ensure that user_agent_map is getting populated correctly for events in the Android beta release cohort.

Acceptance Criteria

  • Data returned from stream tables include accurately populated user_agent_map property
  • Data from the user_agent_map is available in the agent data object

Event Timeline

cjming moved this task from Wikistats Backlog to Data Products Sprint 09 on the Data Products board.
cjming set the point value for this task to 2.
phuedx triaged this task as Unbreak Now! priority.Feb 13 2024, 12:54 PM

Change 1003564 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[schemas/event/secondary@master] Update app base schema

https://gerrit.wikimedia.org/r/1003564

Per discussion with @SNowick_WMF, Shay is ok with having the AgentData object contain the values she usually extracts from the user_agent_map data.

I've made some updates to the Java library to include the full alphanumeric app_version_name (aka wmf_app_version) among other improvements.

Once the following patches/MRs are merged:

I will publish a new release and make a PR against the Android repo.

@Sfaci @SGupta-WMF would either of you mind reviewing/merging https://gerrit.wikimedia.org/r/c/schemas/event/secondary/+/1003564 while @phuedx is out?

Once that is merged, I'll run some tests for https://gitlab.wikimedia.org/repos/data-engineering/metrics-platform/-/merge_requests/32 and post some screenshots for your review of that MR too. I'm hoping these can get into Android's next beta release. Thanks!

Change 1003564 merged by jenkins-bot:

[schemas/event/secondary@master] Update app base schema

https://gerrit.wikimedia.org/r/1003564

Change 1003810 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[schemas/event/secondary@master] Bump app base version to 1.2 for 2 product metrics mobile schemas

https://gerrit.wikimedia.org/r/1003810

Change 1003810 merged by jenkins-bot:

[schemas/event/secondary@master] Bump app base version to 1.2 for 2 product metrics mobile schemas

https://gerrit.wikimedia.org/r/1003810

VirginiaPoundstone lowered the priority of this task from Unbreak Now! to High.

Validating that we are see requested data values in agent column. However before we resolve this I want to request possibly getting device_family data added in. I generally don't NEED that data but while we are investigating anomalies I am not able to check if the missing data is from devices we may not be seeing. As discussed with @cjming this data is available with events and can be passed along in the agent column.

For device_family example see below from MEP data:
MEP user_agent_map values:
{'os_family': 'Android', 'browser_major': '10', 'os_minor': '-', 'os_major': '10', 'wmf_app_version': '2.7.50481-r-2024-04-01', 'device_family': 'Samsung SM-G960F', 'browser_family': 'Android'}