Here's some network performance data pulled from Media Viewer users: P172 Plotted in a google doc: https://docs.google.com/spreadsheets/d/1KpsPgr0XyyQUXkERdYkowYPE1g0C_hmt0kV5cUKKhSQ/edit?usp=sharing
I've discovered this strange phenomenon whereby thumbnails recently added to varnish perform worse on average than thumbnails that have been in varnish for longer. The pattern is obvious once plotted. The performance is as much as 40% worse when the thumbnail has been recently added to varnish. The issue seems to go away once the thumbnail has been in varnish for a day.
We should find out why that's happening and if there's anything we can do about it.