We'll need to get https://github.com/ua-parser/uap-python available on hafnium.

While there, I'd like to request installing it on terbium as well so that we can reliably test and iterate our webperf scripts there before deploying on hafnium (see also T112660).

BTW, don't forget you can now consume (using eventlogging code or not) JSON directly from the Kafka eventlogging_NavigationTiming topic :)

Hm, also, this might not be a good idea, but could be worth considering: should eventlogging-processor do this for all client-side events so that the parsed UA is easily available for anyone to use? MAYYYBE maybe not :) Probably would make the processor work much harder ¯\_(ツ)_/¯

In T112594#1660276, @Ottomata wrote:

Hm, also, this might not be a good idea, but could be worth considering: should eventlogging-processor do this for all client-side events so that the parsed UA is easily available for anyone to use? MAYYYBE maybe not :) Probably would make the processor work much harder ¯\_(ツ)_/¯

It'd certainly simplify things a bit, but it may be better to merely make it available as a method for subscribers (rather than preprocess ahead of time). That way subscribers like Multimedia and NavTiming could do something like:

import eventlogging

events = eventlogging.connect( .. );
for meta in events.filter(schema='Edit', uaMap=True):
    event = meta['event']
    uaMap = meta['uaMap']
    print uaMap['browser']['family']
    print uaMap['browser']['major']

Though it's hardly advantageous compared to using the library directly:

import eventlogging
import user_agent_parser

events = eventlogging.connect( .. );
for meta in events.filter(schema='Edit'):
    event = meta['event']
    uaMap = user_agent_parser.parse(meta['userAgent'])
    print uaMap['user_agent']['family']
    print uaMap['user_agent']['major']

The main hurdle is getting uap-python packaged. Whether to install it on hafnium or eventlog1001 shouldn't matter much.

Change 254355 had a related patch set uploaded (by Ori.livneh):
webperf: log per-user-agent Navigation Timing data

https://gerrit.wikimedia.org/r/254355

Change 254355 merged by Ori.livneh:
webperf: log per-user-agent Navigation Timing data

https://gerrit.wikimedia.org/r/254355

A few ideas:

• Some of the iOS_%s_WebView and iOS_%s_other entries contain periods in the key segment causing them to clobber the space for Navigation Timing properties.
• Let's break down browsers by version inside family to make it easier to group metrics from the same browser (e.g. navtiming.Chrome_Mobile.46.firstPaint).
• Rename by_platform to by_browser. Also to avoid confusion with the $Platform variable we use for desktop vs mobile.
• Rename the browsers to be more consistent with naming scheme of ua-parser per https://github.com/ua-parser/uap-core/blob/HEAD/regexes.yaml. E.g. "Chrome Mobile" (not Mobile Chrome), "Chrome Mobile iOS" (not CriOS), "Mobile Safari", "Mobile Safari WebView" etc. These names are more familiar to us I think, and also used by Analytics.
• Maybe add a generic "Other" so that we're future-proof against what proportion of browsers is outside the ones we have regexes for.

Change 271307 had a related patch set uploaded (by Krinkle):
navtiming: Improve parse_ua and add unit tests

https://gerrit.wikimedia.org/r/271307

Change 271307 merged by Ori.livneh:
navtiming: Improve parse_ua and add unit tests

https://gerrit.wikimedia.org/r/271307