Page MenuHomePhabricator

Improve error handling in Proton tests
Open, MediumPublic

Description

Most Proton tests involve launching the server and sending requests to it via preq. Given the fairly minimal public error reporting in node-template-based applications (and even most of that being discarded by the tests), basically all you end up with is a boolean flag of whether the test passed or failed. That's not very helpful, especially for test errors which are locally not reproducible. Either the logs should be preserved as a test artifact, or the server should output more diagnostic details in test mode and the tests should print it on failure.

Event Timeline

Jhernandez subscribed.

Maybe it is just a case of enabling console logging for the proton instrumentation/logging when running the servers for test.

@Tgr feel free to up the priority if you think this is important.

Yeah, possibly all that this takes is changing [[https://github.com/wikimedia/mediawiki-services-chromium-render/blob/6ba6069eb46e803f9249226dbca0222c34b3cac3/test/utils/server.js#L31|logStream()]] to logStream(true). Not sure whether that would render CI logs unreadable, though. Also I'm not sure where the console ends up, since the server is run via service-runner (so probably a sub-process of some kind?).

As for priority, not sure. I found debugging not locally reproducible issues kind of painful, but it turned out to be sort of PEBKAC in the end (T217113: MediaWiki-Vagrant should use the same Node.js version as Wikimedia production), so maybe these kinds of problems do not occur normally.