Page MenuHomePhabricator

Fix data retrieval scripts using raw user agent strings
Closed, ResolvedPublic

Description

Portal Dashboard is down due to the removal of raw user agent strings from EventLogging. Some other dashboards may be affected as well and need to be identified.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 4 2017, 6:36 PM
debt triaged this task as Unbreak Now! priority.Apr 4 2017, 10:08 PM

We need to get this fixed quickly - the update to the user agent strings knocked down our portal dashboard unexpectedly.

Restricted Application added subscribers: Jay8g, TerraCodes. · View Herald TranscriptApr 4 2017, 10:08 PM

Change 346655 had a related patch set uploaded (by Chelsyx):
[wikimedia/discovery/golden@master] Get browser info from new userAgent field

https://gerrit.wikimedia.org/r/346655

This patch fixed the bug in portal/user_agents.R. After the data is backfilled (We should backfill data from March 29), portal dashboard should be able to work again.

There are other scripts in golden that are affected by the raw userAgent strings removal: search/load_times.R, search/app_event_counts.R and portal/app_link_clicks.sql. Since the version of MariaDB we're using doesn’t support JSON functions, and the queries in these scripts can still accomplish the task, I think there is no need to fix them now. @mpopov please let me know if you think otherwise.

There are other scripts in golden that are affected by the raw userAgent strings removal: search/load_times.R, search/app_event_counts.R and portal/app_link_clicks.sql. Since the version of MariaDB we're using doesn’t support JSON functions, and the queries in these scripts can still accomplish the task, I think there is no need to fix them now. @mpopov please let me know if you think otherwise.

Correct! While those scripts/queries do make use of userAgent field, they are not affected by the change. The only truly affected one is portal/user_agents.R Reviewing your patch now, thanks! :)

Change 346655 merged by Bearloga:
[wikimedia/discovery/golden@master] Get browser info from new userAgent field

https://gerrit.wikimedia.org/r/346655

debt closed this task as Resolved.Apr 7 2017, 7:33 PM
debt awarded a token.