Page MenuHomePhabricator

Continuously run tests using Chrome against WebPageReplay
Closed, ResolvedPublic

Description

Event Timeline

Peter renamed this task from Lets run tests using Chrome against WebPageReplay to Continuously run tests using Chrome against WebPageReplay.Apr 7 2021, 7:40 PM

I've started by comparing runs running WebPageReplay on a Raspberry Pi 4 vs my personal desktop computer (A Mac from 2018) and then run tests on a Moto G5. I've focused on the standard deviation on TTFB,

Raspberry pi

11 runs

[2022-01-14 10:56:26] INFO: https://en.m.wikipedia.org/wiki/Barack_Obama 30 requests, TTFB: 318ms (σ17.00ms), firstPaint: 1.06s (σ147.00ms), FCP: 1.06s (σ147.00ms), DOMContentLoaded: 5.00s (σ970.00ms), LCP: 1.06s (σ147.00ms), CLS: 0.0741 (σ0.00), TBT: 3.62s (σ43.00ms), Load: 5.77s (σ89.00ms) (11 runs)

21 runs

[2022-01-14 12:19:45] INFO: https://en.m.wikipedia.org/wiki/Barack_Obama 30 requests, TTFB: 283ms (σ10.00ms), firstPaint: 1.04s (σ130.00ms), FCP: 1.04s (σ130.00ms), DOMContentLoaded: 5.15s (σ727.00ms), LCP: 1.04s (σ130.00ms), CLS: 0.0741 (σ0.00), TBT: 3.63s (σ46.00ms), Load: 5.77s (σ82.00ms) (21 runs)

Mac

11 runs

[2022-01-14 12:36:21] INFO: https://en.m.wikipedia.org/wiki/Barack_Obama 30 requests, TTFB: 136ms (σ6.00ms), firstPaint: 887ms (σ123.00ms), FCP: 887ms (σ123.00ms), DOMContentLoaded: 5.14s (σ119.00ms), LCP: 887ms (σ123.00ms), CLS: 0.0741 (σ0.00), TBT: 3.62s (σ38.00ms), Load: 5.61s (σ98.00ms) (11 runs)

21 runs

[2022-01-14 13:04:08] INFO: https://en.m.wikipedia.org/wiki/Barack_Obama 30 requests, TTFB: 136ms (σ9.00ms), firstPaint: 930ms (σ125.00ms), firstVisualChange: 945ms (σ125.00ms), FCP: 930ms (σ125.00ms), DOMContentLoaded: 5.31s (σ116.00ms), LCP: 930ms (σ125.00ms), CLS: 0.0741 (σ0.00), TBT: 3.73s (σ76.00ms), Load: 5.81s (σ94.00ms), speedIndex: 1.09s (σ70.00ms), visualComplete85: 1.04s (σ40.00ms), lastVisualChange: 5.39s (σ245.00ms) (21 runs)
[2022-01-14 13:04:12] INFO: HTML stored in /Users/peter/git/sitespeed.io/osx/sitespeed-result/en.m.wikipedia.org/2022-01-14-12-48-25

Next week I plan to keep they running for a couple of hours to see what that looks like.

I did some more tests and it looks like this will work just fine.