Code mirror registers itself with VisualEditor.PluginModules in extension.json. This means it loads along with the editor unconditionally - even though it is used in a minority of VE sessions (user must have VE source mode enabled as well as syntax highlighting).
The CodeMirror v6 packages are 190k, which is not insignificant on slower mobile connections.
We should be loading minimal code on init (toolbar tools etc.), and everything else lazily at least when source mode is started or ideally when highlighting is started.