Page MenuHomePhabricator

MW Core's Jest test suite is very slow
Open, Needs TriagePublic

Description

As of the time of writing, the MediaWiki Core Jest test suite executes 48 tests in about ~20 seconds (~0.4 secs per test). Compare that to the Core QUnit test suite, which runs 797 tests in about ~4.3 seconds (~.005 secs per test). Clearly writing Jest tests the way we have been will not scale.

We should take steps to improve the speeds, while not hurting the breadth of coverage.

Related: T388059: Increase test coverage of Codex Special:Block. The coverage report by itself adds about ~5 seconds to the runtime.

Possible improvements:

  • Increasing the number of workers
  • Bump versions of Jest and other test utilities we use
  • More aggressive mocking
  • Tests of Vue components should use shallowMount instead of mount

Event Timeline

Change #1131118 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/core@master] jest.config.js: limit to searching only in tests/jest

https://gerrit.wikimedia.org/r/1131118

Change #1131118 merged by jenkins-bot:

[mediawiki/core@master] jest.config.js: limit to searching only in tests/jest

https://gerrit.wikimedia.org/r/1131118

I collected a profile for SpecialBlock.test.js and loaded it into speedscope: https://www.speedscope.app/#profileURL=https://tstarling.com/stuff/speedscope/SpecialBlock.test.js.prof.json

It seems to spend most of its time in jsdom. 59% in Window.getComputedStyle() alone. That's mostly called by Codex -- the exact callers are obfuscated because I used the minified Codex package.

jest profile.png (1,836×838 px, 326 KB)

Shallow mounting should help with that, as it wouldn't then run any of the Codex code at all. But a lot of our Jest tests are deliberately not mocking to achieve what are essentially integration tests, giving more confidence. We'll need to find the right balance here.

In our test suite, I think this mainly an issue for SpecialBlock.test.js (which loads everything and its subcomponents). I will focus optimization efforts there for now.