Most modern browsers (with the exception of IE and Chrome) include a back/forward cache, meaning they "freeze" the current state of the page (DOM, JS variables and all) into memory when navigating away, and pull it out from there when using back-button navigation, to speed up navigation by skipping page setup. This functionality can be disabled by various heuristics, if the browser guesses that the page is doing something that would be broken by the caching. Sometimes this disabling can be avoided with some care (e.g. using a pagehide handler instead of an unload handler).
bfcache seems to be disabled on MediaViewer pages (possibly on all MediaWiki pages, I haven't checked). This results in a significant slowdown; we should look into ways of avoiding it if possible.
See this StackOverflow post for various bfcache docs: