Some immature thoughts: Since CodeMirror6 prefers a large bundle of all possibly used resources, we may have to dynamically generate (part of) the source scripts to determine which legacy language modes should also be imported in the bundle.

For example, we can generate one script (pluginModules.js) like this:

// php legacy mode is unavailable
export { css } from '@codemirror/legacy-modes/mode/css';
export { javascript } from '@codemirror/legacy-modes/mode/javascript';

And then in another script (e.g., codemirror.mode.mediawiki.js),

import * as pluginModules from './pluginModules';

for ( const [ language, parser ] of Object.entries( pluginModules ) ) {
    Object.defineProperty( CodeMirrorModeMediaWiki.prototype, language, {
        get() {
            return parser;
        }
    } );
}

I haven't gotten that far, but my thoughts were to use the Lezer packages for other languages, at least when the CM instance is only dealing with one language. I.e. if we were to replace CodeEditor (which I hope to do), we wouldn't use CodeMirrorModeMediaWiki at all, or even WikiEditor for that matter. Just load the Lezer package, and instantiate the EditorView. Lezer packages won't use our styling, and will be more performant than the stream parser implementation. At any rate, I'm saving that effort for later.

For now, I'm trying to just focus on getting the wikitext-based TagModes or simple tag registration working, as with Cite, MediaWiki-extensions-Phonos, TemplateStyles, etc.

<nowiki> and <pre> are currently registered as TagModes in CM5, but I don't see the need for that in CM6. Rather, we just treat them as such and bundle the logic into our stream parser, as opposed to pretending they're not actually part of the MediaWiki language. Patch incoming for that!

Change 987489 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/CodeMirror@master] CodeMirrorModeMediaWiki: add highlighting for <nowiki> and <pre>

https://gerrit.wikimedia.org/r/987489

In T348684#9433721, @MusikAnimal wrote:

I haven't gotten that far, but my thoughts were to use the Lezer packages for other languages, at least when the CM instance is only dealing with one language. I.e. if we were to replace CodeEditor (which I hope to do), we wouldn't use CodeMirrorModeMediaWiki at all, or even WikiEditor for that matter. Just load the Lezer package, and instantiate the EditorView. Lezer packages won't use our styling, and will be more performant than the stream parser implementation.

In this case, do you suggest multiple ~300kB bundles, e.g., one for Wikitext and the other for JavaScript/CSS? By the way, as far as I know, there is no Lezer package for Lua.

My hope was to use dynamic imports so the necessary PluginModules can be loaded at runtime, but I haven't gotten this to work yet. In the end, it may not matter much; We don't have many use cases for mixed languages within wikitext, anyway. SyntaxHighlight could use CodeMirror, but it has a backend implementation, so I don't think we'll integrate CodeMirror there. As a practical example, take MediaWiki-extensions-Score which doesn't yet have a mode. We could write one (it'd have to be a StreamParser I think, not Lezer), and since we know Score is used before page load, Score can use a PHP hook to force CodeMirror to load i.e. CodeMirrorModeScore, which extends CodeMirrorModeMediaWiki, adding in the eatScoreTag() method or whatever and providing it as the tokenizer. So essentially Score tells CodeMirror to use a different endpoint, but no code is duplicated, and caching is consistent for that page. Essentially I suppose this still boils down to what you say -- multiple bundles, but the difference is the other extension provides the customized bundle, rather than tossing it all in CodeMirror. In the end, the storage footprint should be about the same if not lower than what we have now (hopefully!).

Another idea which mayyybe will work is to use the mw.hook system. If we pass around the scope, it seemingly won't matter if the code is built separately. So instead of Score extending CodeMirror internals (and thus delivering a separate package with the whole CM library), it ships one smaller package using just hooks. That might actually be better... if it works.

I do realize PHP is not available as a legacy stream parser, which means MediaWiki-extensions-PhpTags will lose its CM integration. I feel bad about that, as it stomps on the original developer's integration, but I see no way around it sadly. It seems PhpTags isn't maintained much anymore as it is. It emits warnings on REL1_41.

As for pages with different content types, I'd say it's similar to the similar to the MediaWiki-extensions-Score example, where Scribunto could extend just the CodeMirror class (or use hooks), and we leverage all the fancy code editing features that CM provides as extensions, and solve things like T261118: Save code editor settings / preferences across pages in the process. A pity that Lua isn't available as Lezer, but we can still use the legacy stream parser. This again would live in the extension repo, not in CodeMirror. It should be drastically smaller, as opposed to loading it alongside WikIEditor like we have to do for wikitext. It depends on the use case, I guess. T301615: Show syntax highlighting on View Source/protected pages should be easy peasy and lightweight as it doesn't require even an editor. Replacing CodeEditor is separate effort that needs more thought. I want to do it, but it's outside the scope of the CM6 upgrade project and more just a personal interest of mine. I do want to deliberately leave room for it, and avoid making it harder to do down the road, so keep sharing your thoughts! :)

Change 987489 merged by jenkins-bot:

[mediawiki/extensions/CodeMirror@master] CodeMirrorModeMediaWiki: add highlighting for <nowiki> and <pre>

https://gerrit.wikimedia.org/r/987489

Change 989232 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/CodeMirror@master] CodeMirrorModeMediaWiki: add support for mediawiki TagModes

https://gerrit.wikimedia.org/r/989232

I'm probably missing something obvious, but it seems like there's special handling for ampersands, e.g. <nowiki>&</nowiki>. This gives an "Unknown highlighting tag undefined" error.

Change 989232 merged by jenkins-bot:

[mediawiki/extensions/CodeMirror@master] CodeMirrorModeMediaWiki: add support for mediawiki TagModes

https://gerrit.wikimedia.org/r/989232

I guess this is due to type coercion from undefined to 'undefined'. Maybe returning an empty string will solve this problem.

Mixed languages within wikitext is tracked at T357480, but I'm not considering that a "must" for the CM6 upgrade.