Page MenuHomePhabricator
Create Task
Maniphest T99932

extracting Mobile App info in webrequest {hawk} [8 pts]
Closed, ResolvedPublic
Actions

Description

Discussion notes on counting pageviews for mobile apps: http://etherpad.wikimedia.org/p/PV_Analytics_Infra

What would be the most effective way to extract the info after the slash in:

WikipediaApp/2.0-r-2015-04-23 (Android 5.0.1; Phone) Google Play
WikipediaApp/4.1.2 (iPhone OS 8.3; Tablet)

So that it can be easily queried against? We could add it to the refined tables or simply use regex in Hive to get what's needed.

Based on tasking meeting http://etherpad.wikimedia.org/p/analytics-tasking,
It makes sense to add it to the map of the UA field
Work:

  • make special case in wrapper of UA parser to handle this

Details

SubjectRepoBranchLines +/-
Add 2 fields and 1 field update to refine tableanalytics/refinerymaster+11 -5
Correct UAParser not to fail for malformed wmf_appanalytics/refinery/sourcemaster+12 -3
Add app version to user agent mapanalytics/refinery/sourcemaster+64 -2
Customize query in gerrit

Related Objects
Search...

Event Timeline

• kevinator created this task.May 21 2015, 5:53 PM
• kevinator assigned this task to JAllemandou.
• kevinator raised the priority of this task from to Medium.
• kevinator updated the task description. (Show Details)
• kevinator added projects: Analytics-Kanban, Analytics-Clusters.
• kevinator subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 21 2015, 5:53 PM
• kevinator set Security to None.May 21 2015, 5:55 PM
• kevinator added subscribers: dr0ptp4kt, Dbrant, • Fjalapeno.
• ggellerman subscribed.May 21 2015, 8:24 PM
• kevinator renamed this task from Assess how to extract Mobile App info in webrequest to Assess how to extract Mobile App info in webrequest [8 pts].Jun 1 2015, 4:25 PM
• kevinator reassigned this task from JAllemandou to mforns.
• kevinator moved this task from Next Up to Tasked_Hidden on the Analytics-Kanban board.Jun 1 2015, 4:28 PM
• kevinator renamed this task from Assess how to extract Mobile App info in webrequest [8 pts] to Assess how to extract Mobile App info in webrequest [8 pts] {hawk}.Jun 2 2015, 10:06 PM
• kevinator updated the task description. (Show Details)Jun 2 2015, 10:09 PM
mforns moved this task from Tasked_Hidden to In Progress on the Analytics-Kanban board.Jun 3 2015, 6:09 PM
Milimetric renamed this task from Assess how to extract Mobile App info in webrequest [8 pts] {hawk} to Assess how to extract Mobile App info in webrequest {hawk} [8 pts].Jun 3 2015, 8:55 PM
gerritbot subscribed.Jun 5 2015, 10:28 AM

Change 216060 had a related patch set uploaded (by Mforns):
Add app version to user agent map

https://gerrit.wikimedia.org/r/216060

gerritbot added a project: Patch-For-Review.Jun 5 2015, 10:28 AM
• ggellerman moved this task from In Progress to In Code Review on the Analytics-Kanban board.Jun 5 2015, 3:34 PM
• ggellerman renamed this task from Assess how to extract Mobile App info in webrequest {hawk} [8 pts] to extracting Mobile App info in webrequest {hawk} [8 pts].Jun 9 2015, 3:32 PM
• kevinator added a parent task: T96044: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts].Jun 9 2015, 3:32 PM
gerritbot added a comment.Jun 9 2015, 11:17 PM

Change 216060 merged by Ottomata:
Add app version to user agent map

https://gerrit.wikimedia.org/r/216060

Ottomata mentioned this in rANRS986111c744ae: Add app version to user agent map.Jun 9 2015, 11:18 PM
gerritbot added a comment.Jun 10 2015, 7:47 AM

Change 217226 had a related patch set uploaded (by Joal):
Correct UAParser not to fail for malformed wmf_app

https://gerrit.wikimedia.org/r/217226

gerritbot added a comment.Jun 10 2015, 7:57 AM

Change 217229 had a related patch set uploaded (by Joal):
Add 2 fields and 1 update to refine table

https://gerrit.wikimedia.org/r/217229

gerritbot added a comment.Jun 10 2015, 6:51 PM

Change 217226 merged by Ottomata:
Correct UAParser not to fail for malformed wmf_app

https://gerrit.wikimedia.org/r/217226

JAllemandou mentioned this in rANRSfd3c97efc6d4: Correct UAParser not to fail for malformed wmf_app.Jun 10 2015, 6:52 PM
gerritbot added a comment.Jun 10 2015, 8:13 PM

Change 217229 merged by Ottomata:
Add 2 fields and 1 field update to refine table

https://gerrit.wikimedia.org/r/217229

JAllemandou mentioned this in rANREab6aa979e312: Add 2 fields and 1 field update to refine table.Jun 10 2015, 8:14 PM
JAllemandou moved this task from In Code Review to Ready to Deploy on the Analytics-Kanban board.Jun 11 2015, 10:28 AM
JAllemandou moved this task from Ready to Deploy to Done on the Analytics-Kanban board.Jun 11 2015, 10:39 AM
• kevinator closed this task as Resolved.Jun 11 2015, 9:03 PM

verified. Example user_agent_map found in the logs:
{"browser_major":"4","os_family":"Android","os_major":"4","device_family":"Other","browser_family":"Android","os_minor":"4","wmf_app_version":"2.0.102-r-2015-05-14"}

dr0ptp4kt mentioned this in T97355: Pageviews by UA and form factor.Nov 25 2015, 7:43 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL