The job `mediawiki-core-code-coverage-docker` usually takes 2h / 2h30 based on https://integration.wikimedia.org/ci/job/mediawiki-core-code-coverage-docker/buildTimeTrend
integration-slave-docker-1054 is usually less than two hours. The job times out after four hours on 1055 and 1056 :(
After some investigations below, the slow instances are on cloudvirt1005 / cloudvirt1006 which have `Intel Xeon E3-12xx v2 (Ivy Bridge)` at 2.7Ghz. The fasted builds are on instances having a Skylake processor at 2.3Ghz.
I can imagine a newer architecture offers improvement, but a for a single thread CPU bound doing simple maths, I would expect the Ivy Bridge Xeon at 2.7Ghz to be faster than the Skylake one at 2.3Ghz.
I tried a couple small CPU benchmarks which I ran on instances:
* Compute 10k prime numbers using 64 bit with the `sysbench` package and running: `sysbench --test=cpu run`.
* Do some basic math with `stress-ng` and running: `stress-ng --cpu 1 --cpu-ops=400000`
Results:
| Model | Intel Xeon E3-12xx v2 (Ivy Bridge) | Intel Core Processor (Skylake)
|--|--|--
| cpu MHz | 2,7 Ghz | 2,3 GHz
| bogoMips | 5390 | 4590
| sysbench duration | ~ 16 seconds | ~ 9 seconds
| stress-ng | ~ 18 seconds | ~ 10 seconds
Rearranged, not taking in account the turbo speed:
| Host | Model | base speed | bogoMips | Sysbench
|---------------|-----------------|------------|----------|---------
| cloudvirt1006 | Xeon E5-2697 v2 | 2,700 MHz | 5,386 | **15.13s**
| cloudvirt1025 | Xeon Gold 6140 | 2,300 MHz | 4,590 | **9.24s**
| cobalt | Xeon E5-2623 v3 | 3,000 MHz | 6,000 | **9,30s**
| contint1001 | Xeon E5-2640 v3 | 2,600 MHz | 5,200 | **9.37s**
| @hashar | i7-8550U | 1,800 MHz | 4,000 | **7,43s**
Note how despite the bogoMips and CPU speed being higher on the Xeon Ivy Bridge, it performs twice slower.
I really dont get why the Intel Xeon is so slow :-/ Maybe it is an oddity due to kvm or a BIOS / hardware configuration issue. One would have to run the same benchmarks on the real servers for comparison?