That company has a good price and we can re-use old bare metal servers. Lets try them out as a first step.
Description
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Resolved | Peter | T307446 Move to bare metal servers for performance tests | |||
Resolved | Peter | T311980 Move tests from AWS to bare metal | |||
Resolved | Peter | T312203 Test running synthetic performance test at hetzner |
Event Timeline
I tried a instance this morning in Finland but it was unusable because the TTFB was too unstable even when throttling the connection:
[2022-07-06 12:23:05] INFO: https://en.wikipedia.org/wiki/Barack_Obama 96 requests, TTFB: 738ms (σ126.00ms), firstPaint: 1.58s (σ120.00ms), firstVisualChange: 1.63s (σ124.00ms), FCP: 1.58s (σ120.00ms), DOMContentLoaded: 1.62s (σ120.00ms), LCP: 1.58s (σ120.00ms), CLS: 0 (σ0.00), TBT: 75ms (σ14.00ms), Load: 2.53s (σ148.00ms), speedIndex: 1.63s (σ125.00ms), visualComplete85: 1.63s (σ124.00ms), lastVisualChange: 1.70s (σ130.00ms) (21 runs)
However it could be that I accidentally got the lowest internet connection for that server. I'm gonna try again for a server hosted in Germany.
I tested on Ubuntu 20.04 and installed the necessary code. Everything was super clean except that the variation in TTFB was a couple of 100 ms. That should be just a couple of ms. First I thought something was wrong with TC but I verified everything manually and it looked like it was working and I got faster/slower connection depending on the settings. I've contacted customer support to ask for guidance when I try on another server.
Something was broken with the first server I tried. With the new one it looks like this:
[2022-07-06 15:54:56] INFO: https://en.wikipedia.org/wiki/Barack_Obama 96 requests, TTFB: 524ms (σ2.00ms), firstPaint: 1.12s (σ16.00ms), firstVisualChange: 1.20s (σ17.00ms), FCP: 1.12s (σ16.00ms), DOMContentLoaded: 1.18s (σ27.00ms), LCP: 1.12s (σ16.00ms), CLS: 0 (σ0.00), TBT: 77ms (σ9.00ms), Load: 1.72s (σ14.00ms), speedIndex: 1.20s (σ16.00ms), visualComplete85: 1.20s (σ17.00ms), lastVisualChange: 1.24s (σ19.00ms) (21 runs)
The first server had stddev 126 ms, the second 2 ms.
I deployed on two servers and tried running tests continuously on Ubuntu. I did 5 runs per URL and maybe that was too few, here's a diff for 100 ms:
But I think we can work with that.
CPU variation looked good, the TTFB had a couple of hiccups:
I think we can move on and test them and then run with WebPageReplay too,