| Peter | |
| Feb 4 2021, 9:38 AM |
| F34107300: Screenshot 2021-02-16 at 13.29.37.png | |
| Feb 16 2021, 12:36 PM |
| F34107298: Screenshot 2021-02-16 at 13.30.08.png | |
| Feb 16 2021, 12:36 PM |
| F34107302: Screenshot 2021-02-16 at 13.29.57.png | |
| Feb 16 2021, 12:36 PM |
| F34107296: Screenshot 2021-02-16 at 13.28.27.png | |
| Feb 16 2021, 12:36 PM |
| F34095079: Screenshot 2021-02-08 at 08.33.44.png | |
| Feb 8 2021, 1:03 PM |
| F34095077: Screenshot 2021-02-08 at 08.34.23.png | |
| Feb 8 2021, 1:03 PM |
| F34091978: Screenshot 2021-02-04 at 11.19.12.png | |
| Feb 4 2021, 10:23 AM |
| F34091975: Screenshot 2021-02-04 at 11.18.27.png | |
| Feb 4 2021, 10:23 AM |
According to Pat Meenan the new Mac M1 chip will produce the same web performance results as running on an actual iPhone.
Running on a Mac instead will make it much easier to have a throttled connection (throttling the network directly on the Mac instead of using a throttled wifi). It will also be lower cost and we can try on many different devices if we want (or at least one iPhone and one iPad).
We need to try it out and see how stable metrics we get and let it run for a week or two.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Declined | Peter | T268432 Add iOS device for performance testing | |||
| Declined | Peter | T269522 Setup iOS device at Kobiton | |||
| Resolved | Peter | T273855 Evaluate using the iOS simulator on a Mac M1 to run web performance tests | |||
| Resolved | Peter | T273856 Explore running desktop test on Mac |
The test we run is using iPad (8th generation) and a iPhone 12 Pro Max.
So far the tests looks great. We do 5 runs. For iPhone I just throttled 3g and for the iPad throttled 4g.
The metrics for the iPad:
And for the iPhone:
There's one bump on the iPhone tests and that time a couple of the runs was skewed so the median was made out of too few runs. The problem was that the test ended before the page has finished loading. I need to fine tune some settings.
Metrics running on a iPhone is super stable (the same stability as we get running using WebPageReplay on AWS).
Iphone:
For the iPad with larger screen three earlier first paint,. When I look at the variance between runs its much higher on the iPad than iPhone, so we should probably do more runs for the iPad.
Ipad
The price running on an Mc Mini is much cheaper that running on our current hosting solution.
Today I increased number of runs to 11 for the iPad and added two more URLs for the iPhone tests juts to verify that everything is ok.
Increased number of runs for the iPad helped:
I've also added more URLs for the iPhone tests, lets look at them:
This worked really well and I think best case we can run on a Mac mini M1 + have one old iPhone attached to that so we can run both test cases.