The Jenkins jobs triggered for oojs/ui.git takes too much resources on the CI infrastructure, specially when lot of changes are made in a serie and merged.
So if one send a patch and +2 it that is six Jessie instances being busy for 6 minutes or four during 9-10 minutes. It only takes three such patches to consume the whole pool.
A serie of 30 patches got merged on March 14th 2016 between 19:00UTC to midnight and from 17:50UTC to midnight that is 241 jobs and thus 241 instances consumed. That is too much for the CI infrastructure to handle.
When patches are proposed for the oojs/ui.git repository, we trigger three different jobs that run:
- npm test : grunt test
- npm demos : grunt publish-build demos
- npm doc : grunt build && jsduck && copy:jsduck
All of them share the CI setup overhead to clone the repository. They then run npm install which probably takes half of the build time.
Each also invoke composer install, though it benefits from caching.
Looking at the grunt tasks being run, each job also share common tasks some being rather slow (ex: colorizeSvg, svg2png:dist).
Using the job console output:
vimdiff <(grep 'Running' npm-test.log) <(grep 'Running' npm-run-demos.log) <(grep 'Running' npm-run-doc.log)
What would be nice
- I would like all three build steps (test, doc, demos) to be unified in a single Jenkins job. That would clear out the overhead of git clone / npm install.
- We probably want a new task defining the tasks to run, paying attention to avoid running twice the slowest one (svg2png and colorizeSvg).
We can have Jenkins to invoke a specific npm entry point such as npm run jenkins which would have the appropriate pre/post task and maybe invoke a specific/custom grunt task.