Anonymous users have mTouched set to "0" meaning that they have never been modified. But wfTimestamp() treats this value as "now" causing the exact opposite. The consequence is that static resources loaded under anonymous user are never cached.
This looks like a clear bug, even the source code comment suggest that this value should cause caching of any page. The attached patch fixes this.
This is network communication before: http://www.vrana.cz/test/mediawiki/before.png and after: http://www.vrana.cz/test/mediawiki/after.png. The difference between 1.59 s and 2.63 and the 304 status are important.