Page MenuHomePhabricator

Allow stackTraces inside MobileFrontend error reporting
Closed, DeclinedPublic

Description

During code review many concerns came up relating to URI length being exceeded, particularly due to long page titles and the length of the stack trace.

Given the amount of discussion in code review around this subject, it's clear we need to do further investigation here.

Given we have no data to work on, we should probably investigate how many errors we are getting and what the current URI length of those errors are before continuing further.

Acceptance criteria

Event Timeline

I'm wondering what is the best way to store as much as possible without crossing the hard 1k limit for the URL. Adding some hard limit is a good way to go, but there are two pitfalls:

  • those strings will be encodeURIComponent(), which means that most probably the output will be longer, for example, the userUrl can have non-latin characters. if that happens the encoded string will be much longer than the not encoded one
  • some fields like errorUrl might be empty, we can use that space for longer stack traces
  • error messages can be super short (like Type error.) or pretty long (over 200 characters)
  • stacktrace might contain repetitive data (like most of the files share a similar path)

I came with two ideas:

Use the remaining space in smart way

We should create a hard length limit, but for all fields with variable length. I assume that sessionToken, skin, mobileMode, isAnon will have predefined length (like sessionToken is 20 chars or max length of 4 to 7 chars for mobileMode.
Also, those fields do not contain non-latin characters, we can assume that the length of that part will be <100chars). We should worry about fields like userUrl, errorUrl, errorMessage, stack, as those can be pretty long. Every time we want to store another property, we will have to think how many characters do we have, sth like:

var payload = {...} 
var remaining = 800;
   
payload.userUrl = userUrl.slice(0, remaining);
remaining = Math.max(0, remaining - encodeURIComponent(payload.userUrl).length);
payload.errorUrl = errorUrl.slice(0, remaining);
remaining = Math.max(0, remaining - encodeURIComponent(payload.errorUrl).length);
payload.errorMessage = errorMessage.slice(0, remaining);
remaining = Math.max(0, remaining - encodeURIComponent(payload.errorMessage).length);
payload.stack = stack.slice(0, remaining);

With that approach, each time we push new property to our payload, we reduce the remaining by the encoded property size. Of course, this approach is not perfect, as the encoded string might not fit into remaining space. But that is easily fixable by creating a function that encodes the string, checks the length, and if it doesn't fit shortens it by one character, and tries to fit encoded string once again (in a loop).

Compress data

The second (totally different) approach would be to compress the payload, there are JS libs that allow us to compress text using for example LZW. the errorUrl, userUrl and stack will contain many similar substrings, also the errorMessage can share some strings (like path or filename). After a quick google, I found the first lib that offers us to compress strings - ls-string.js, and whats interesting, it offers LZString.compressToEncodedURIComponent(string); which can be really helpful in our case. Although it can help us send more data to the analytics server, it will require processing (decompresssion) - most probably on varnish. Probably because of that, we won't be able to go with that option.