Our metrics (esp. this one) indicate that unprefetched loading time for MediaViewer is on par with the loading time of the main image on the file page (despite showing significantly larger files), so this is not a priority, just a collecting place for ideas.
The median loading time (click-to-display time) of MediaViewer is 1400ms; the median time it takes MediaViewer to fetch a thumbnail is 600ms. That's a significant gap and suggests that further optimization should be possible.