Find a way to detect when error reporting breaks.
- write an extension that produces various types of errors (PHP, JS module setup, JS async callback etc)
- ???
- profit
Step 2 can probably be done via Selenium, but need to think about that more.
Description
Description
Details
Details
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Initial commit | mediawiki/extensions/Buggy | master | +595 -0 |
Event Timeline
Comment Actions
Change 188963 had a related patch set uploaded (by Gergő Tisza):
Initial commit
Comment Actions
Karma seems the most appropriate for this: it executes QUnit tests in an actual browser (ie. much higher level of abstraction than hand-coding JS tests in Watir), and is already included in the MediaWiki CI suite. QUnit has its own onerror handler, but that can just be disabled for the duration of these tests. sinon.js's mocking of setTimeout etc. also needs to be disabled. These tests would be integration tests rather than unit tests, making this a slight abuse of QUnit, but meh.
Comment Actions
Backlogging. As long as we are not going to do any of the complicated callback wrapping, setting up an automated test is not really worth the effort.
Comment Actions
Removed patch-for-review since the patch is merged. Removed browser-tests because it looks to me it is not related. Please add the tag(s) back if needed.
Comment Actions
I was hoping that browser test could provide the ??? part :)
I'm not really sure how to go about this. We could set up a browser test to check that an error gets reported when the browser visits page X which is known to have a bug, but that's not terribly interesting - error catching is done via the onerror event which we don't really expect to fail, and there are QUnit tests to make sure that the custom error reporting does not break when the error event is fired.
What's actually worth testing is that a sane amount of information is extracted (such as a stack trace). That depends on a number of interacting components (the browser's own API for error details, ResourceLoader, the Sentry extension, the 3rd party code from Raven-js, HTTP header settings) and could break easily. It does not seem easy to test though, especially if eventually we want to involve source maps, which are applied on the Sentry server. Given that browser behavior is one potential source of problems, the test would have to involve browser testing at the very least (maybe via Karma or something similar and not Watir, but tests would definitely have to run in multiple browsers), but then the errors logs should probably have to queried from the Sentry server for verification...
Comment Actions
Anyway this is a lot of effort and not really worth it unless Sentry gets deployed widely. Maybe not even then since we can just use the actual live error reports to notice when error reporting doesn't work right... not sure.
Comment Actions
This task has been assigned to the same task owner for more than two years. Resetting task assignee due to inactivity, to decrease task cookie-licking and to get a slightly more realistic overview of plans. Please feel free to assign this task to yourself again if you still realistically work or plan to work on this task - it would be welcome!
For tips how to manage individual work in Phabricator (noisy notifications, lists of task, etc.), see https://phabricator.wikimedia.org/T228575#6237124 for available options.
(For the records, two emails were sent to assignee addresses before resetting assignees. See T228575 for more info and for potential feedback. Thanks!)
Comment Actions
At the time of creating this task it was assumed that the server side implementation of error logging would be based on Sentry. We have eventually decided on a different implementation, so de-tagging.
Comment Actions
The parent tasks have been declined, hence also declining this task.
If this task should still be open, then please update its description and associate an active project - thanks!