This is a proposal with the following goals:
- Enable a reliable git cache on our integration slaves. (Blocks T93703, and T96627)
- Increase test execution performance.
- Increase stability by reducing ways that builds can interact with each other.
There are many race conditions and potential transient issues between active workspaces and a local git cache. By decreasing our executor slots from four concurrent workers to one, we can easily have our git cache updated by a scheduled Jenkins job that will naturally only occur between builds.
In order to decrease our executor slots and keep our current build throughout, we'll need to increase the number of slaves quite a bit. Something that will happen anyway when we start using Nodepool. We can also reduce our instance size as we wouldn't need as much CPU/RAM/Disk capacity.
Figuring our the right instance size and number of slaves is a good preparation for our Nodepool setup
In addition this would make our environment more isolated. And it is what OpenStack has been doing for a while (before they switched to Nodepool).