Page MenuHomePhabricator

EL unable to decode may apps events {oryx}
Closed, ResolvedPublic

Description

2015-04-22 06:34:11,522 (MainThread) Unable to decode: ?%7B%22wiki%22%3A%22mlwiki%22%2C%22schema%22%3A%22MobileWikiAppArticleSuggestions%22%2C%22revision%22%3A11448426%2C%22event%22%3A%7B%22readMoreIndex%22%3A0%2C%22action%22%3A%22clicked%22%2C%22pageTitle%22%3A%22%E0%B4%AA%E0%B5%8D%E0%B4%B0%E0%B4%A7%E0%B4%BE%E0%B4%A8%20%E0%B4%A4%E0%B4%BE%E0%B5%BE%22%2C%22version%22%3A0%2C%22appInstallID%22%3A%221b3ef6e1-f336-40bd-b54b-d5c5ecbbe80a%22%2C%22readMoreList%22%3A%22%E0%B4%B1%E0%B5%86%E0%B4%A1%E0%B5%8D%20%E0%B4%B2%E0%B4%BF%E0%B4%B8%E0%B5%8D%E0%B4%B1%E0%B5%8D%E0%B4%B1%E0%B5%8D%7C%E0%B4%95%E0%B5%8D%E0%B4%B5%E0%B5%88%E0%B4%B1%E0%B5%8D%E0%B4%B1%E0%B5%8D%20%E0%B4%AF%E0%B5%82%E0%B4%A3%E0%B4%BF%E0%B4%B5%E0%B5%87%E0%B4%B4%E0%B5%8D%E0%B4%B8%E0%B5%BD%20%E0%B4%B8%E0%B5%BC%E0%B4%95%E0%B5%8D%E0%B4%AF%E0%B5%82%E0%B4%9F%E0%B5%8D%E0%B4%9F%E0%B5%8D%20%E0%B4%B8%E0%B4%BF%E0%B4%AE%E0%B5%81%E0%B4%B2%E0%B5%87%E0%B4%B1%E0%B5%8D%E0%B4%B1%E0%B5%BC%7C%E0%B4%87%E0%B4%97%E0%B5%8D%20%E0%B4%A8%E0%B5%8B%E0%B4%AC%E0%B5%BD%20%E0%B4%B8%E0%B4%AE%E0%B5%8D%E0%B4%AE%E0%B4%BE%E0%B4%A8%E0%B4%82%22% cp1070.eqiad.wmnet 1756655 2015-04-22T06:34:11 10.64.0.107 WikipediaApp/2.0-r-2015-03-23 (Android 4.2.2; Phone) Google Play
(Expecting ',' delimiter: line 1 column 307 (char 306))

2015-04-22 06:34:14,567 (MainThread) Unable to decode: ?%7B%22wiki%22%3A%22arwiki%22%2C%22schema%22%3A%22MobileWikiAppArticleSuggestions%22%2C%22revision%22%3A11448426%2C%22event%22%3A%7B%22readMoreIndex%22%3A2%2C%22action%22%3A%22clicked%22%2C%22pageTitle%22%3A%22%D9%82%D8%B7%D8%A8%20%D8%AC%D9%86%D9%88%D8%A8%D9%8A%22%2C%22version%22%3A0%2C%22appInstallID%22%3A%22915e1c6f-95bd-49d4-ab65-8451333bc669%22%2C%22readMoreList%22%3A%22%D8%A7%D9%84%D9%85%D9%82%D8%A7%D8%B7%D8%B9%D8%A9%20%D8%A7%D9%84%D8%A8%D8%B1%D9%8A%D8%B7%D8%A7%D9%86%D9%8A%D8%A9%20%D8%A8%D8%A7%D9%84%D9%82%D8%A7%D8%B1%D8%A9%20%D8%A7%D9%84%D9%82%D8%B7%D8%A8%D9%8A%D8%A9%20%D8%A7%D9%84%D8%AC%D9%86%D9%88%D8%A8%D9%8A%D8%A9%7C%D8%A7%D9%84%D8%A8%D8%B9%D8%AB%D8%A9%20%D8%A7%D9%84%D9%8A%D8%A7%D8%A8%D8%A7%D9%86%D9%8A%D8%A9%20%D9%84%D9%84%D9%82%D8%A7%D8%B1%D8%A9%20%D8%A7%D9%84%D9%82%D8%B7%D8%A8%D9%8A%D8%A9%20%D8%A7%D9%84%D8%AC%D9%86%D9%88%D8%A8%D9%8A%D8%A9%7C%D8%AD%D9%8A%D9%88%D8%A7%D9%86%D8%A7%D8%AA%20%D8%A7%D9%84%D9%82%D8%A7%D8%B1%D8%A9%20%D8%A7%D9%84%D9%82%D8%B7%D8%A8%D9%8A%D8%A9%20%D8%A7%D9%84%D8%AC%D9% cp3020.esams.wmnet 2188806 2015-04-22T06:34:14 10.20.0.120 WikipediaApp/2.0-r-2015-03-23 (Android 4.4.2; Phone) Google Play
()

Are these events too long? They look like it , probably the are being chuncked by varnish logging limit. Now, this is a problem that needs to be fixed in teh client as we have no way to increase varnish logging limit for urls.

Event Timeline

Nuria created this task.Apr 23 2015, 12:00 AM
Nuria raised the priority of this task from to Needs Triage.
Nuria updated the task description. (Show Details)
Nuria added projects: Analytics-Kanban, Mobile.
Nuria added subscribers: kevinator, Deskana, JAllemandou.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 23 2015, 12:00 AM
kevinator renamed this task from EL unable to decode may apps events to EL unable to decode may apps events {oryx}.May 2 2015, 1:17 AM
kevinator triaged this task as High priority.
kevinator set Security to None.
kevinator added a subscriber: mforns.

@mforns, can you look at this as part of your EL tour of duty next week? I don't know if it's still an issue...
If it is, the easiest solution is to notify the mobile team and see if they can shorten their payload.

mforns moved this task from In Progress to Paused on the Analytics-Kanban board.May 8 2015, 9:44 AM
mforns moved this task from Paused to In Progress on the Analytics-Kanban board.May 11 2015, 10:45 AM

In fact, the events are being truncated for being larger than EL can handle.

The reason of the exceptional length of the events lies not in the size of the schema, which is OK.
But in the field readMoreList, that can get big when it contains special characters like:

Чемпионат мира по хоккею

Which URI encoded is:

%D0%A7%D0%B5%D0%BC%D0%BF%D0%B8%D0%BE%D0%BD%D0%B0%D1%82%20%D0%BC%D0%B8%D1%80%D0%B0%20%D0%BF%D0%BE%20%D1%85%D0%BE%D0%BA%D0%BA%D0%B5%D1%8E

These events represent 1% approximately of all MobileWikiAppArticleSuggestions schema events.
But it seems that they are biased to certain languages...

We are trying hard to improve EL service, but for now the size threshold is a limitation of the system.
Is it possible for you, Mobile team, to store readMoreList in pageIDs or other shorter identifyers?

mforns closed this task as Resolved.May 11 2015, 3:39 PM
mforns moved this task from In Progress to Done on the Analytics-Kanban board.May 12 2015, 3:42 PM