Change #1107081 merged by jenkins-bot:

[mediawiki/extensions/CodeMirror@master] Add syntax highlighting, folding and autocompletion for Lua

https://gerrit.wikimedia.org/r/1107081

Change #1169279 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/CodeMirror@master] config: enable linting of Scribunto namespace by default

https://gerrit.wikimedia.org/r/1169279

Change #1169279 merged by jenkins-bot:

[mediawiki/extensions/CodeMirror@master] config: enable linting of Scribunto namespace by default

https://gerrit.wikimedia.org/r/1169279

Change #1169803 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/TemplateStyles@master] CodeMirrorHooks: add integration with CodeMirror

https://gerrit.wikimedia.org/r/1169803

A quick note that Extension:AbuseFilter has a code editor at pages such as https://en.wikipedia.org/wiki/Special:AbuseFilter/1. Might want to investigate adding that to your list of code editors to replace with CodeMirror.

AbuseFilter uses a custom programming language, so that part might be challenging.

Change #1169804 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/JsonConfig@master] CodeMirrorHooks: add integration with CodeMirror

https://gerrit.wikimedia.org/r/1169804

Change #1169805 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[integration/config@master] zuul: add CodeMirror as Phan dependency for JsonConfig, TemplateStyles

https://gerrit.wikimedia.org/r/1169805

In T373711#11007507, @Novem_Linguae wrote:

A quick note that Extension:AbuseFilter has a code editor at pages such as https://en.wikipedia.org/wiki/Special:AbuseFilter/1. Might want to investigate adding that to your list of code editors to replace with CodeMirror.

AbuseFilter uses a custom programming language, so that part might be challenging.

Also VisualEditor has a widget that uses Ace, and WikiLambda a Vue component.

In T373711#10913806, @Od1n wrote:

Since it hasn't been brought up yet on this ticket, I wanted to mention that the AbuseFilter's rules editor also currently uses Ace Editor. What's particularly noteworthy here is that AbuseFilter rules are their own distinct language, not one of the standard content models being discussed for CodeMirror 6 support.

This means if we fully transition away from Ace, we'll need to consider how this affects the AbuseFilter editor, and whether CodeMirror 6 will eventually support this specialized language for highlighting and linting.

In T373711#11007507, @Novem_Linguae wrote:

A quick note that Extension:AbuseFilter has a code editor at pages such as https://en.wikipedia.org/wiki/Special:AbuseFilter/1. Might want to investigate adding that to your list of code editors to replace with CodeMirror.

AbuseFilter uses a custom programming language, so that part might be challenging.

Sorry the first comment didn't get a reply!

Indeed, AbuseFilter has a custom Ace mode that depends on the ext.codeEditor.ace module, and thus the CodeEditor extension. While this is a bit more involved than the other extensions, I think it's perfectly doable to convert the Ace stream parser to one that works with CodeMirror. At any rate, certainly for a different task. I've filed T399673: Add CodeMirror mode for AbuseFilter syntax.

Also VisualEditor has a widget that uses Ace, and WikiLambda a Vue component.

Indeed. I believe there are other repos as well that we'll eventually want to look into.

This task is primarily about the modes we currently support, and the experimental trial of having Beta feature users use it in place of CodeEditor. Feature completeness across the wikiverse would still be quite a ways away, barring a broader coordinated effort. In the meantime, the extensions should be able to coexist peacefully.

I like the idea of standardizing to CodeMirror from a technical debt perspective. Should be easier to work on if all the code editors are the same :) But yes, won't happen overnight. Thanks for all your hard work on this.

Change #1169805 merged by jenkins-bot:

[integration/config@master] zuul: add CodeMirror as Phan dependency for JsonConfig, TemplateStyles

https://gerrit.wikimedia.org/r/1169805

Change #1170190 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/UploadWizard@master] CodeMirrorHooks: add integration with CodeMirror

https://gerrit.wikimedia.org/r/1170190

Change #1170191 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[integration/config@master] zuul: add CodeMirror as Phan dependency for UploadWizard

https://gerrit.wikimedia.org/r/1170191

Change #1170192 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/Gadgets@master] CodeMirrorHooks: add integration with CodeMirror

https://gerrit.wikimedia.org/r/1170192

Change #1170193 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[integration/config@master] zuul: Add CodeMirror as Phan dependency of Gadgets

https://gerrit.wikimedia.org/r/1170193

Change #1170191 merged by jenkins-bot:

[integration/config@master] zuul: add CodeMirror as Phan dependency for UploadWizard

https://gerrit.wikimedia.org/r/1170191

Change #1170193 merged by jenkins-bot:

[integration/config@master] zuul: Add CodeMirror as Phan dependency of Gadgets

https://gerrit.wikimedia.org/r/1170193

Change #1169803 merged by jenkins-bot:

[mediawiki/extensions/TemplateStyles@master] CodeMirrorHooks: add integration with CodeMirror

https://gerrit.wikimedia.org/r/1169803

Change #1169804 merged by jenkins-bot:

[mediawiki/extensions/JsonConfig@master] CodeMirrorHooks: add integration with CodeMirror

https://gerrit.wikimedia.org/r/1169804

Change #1170190 merged by jenkins-bot:

[mediawiki/extensions/UploadWizard@master] CodeMirrorHooks: add integration with CodeMirror

https://gerrit.wikimedia.org/r/1170190

It looks like we need to check the beta feature in all of the CodeMirrorHooks files. Otherwise, if we have for example the following config, we get neither CodeEditor nor CodeMirror when editing JsonConfig pages:

$wgCodeMirrorEnabledModes['json'] = true;
$wgCodeEditorEnabledModes['json'] = false;
$wgJsonConfigUseCodeEditor = false;
$wgJsonConfigUseCodeMirror = true;

We cannot do this entirely with config logic since checking the beta feature requires access to the request context.

In T373711#10971333, @Bhsd wrote:

Is it possible to keep $wgScribuntoUseCodeEditor=True but give it a lower precedence than $wgScribuntoUseCodeMirror? In this way, CodeEditor can be a fallback if CodeMirror6 is not enabled.

Something like that could work, but it would depend on the loading order of the extensions, which we want to avoid. I think it will be more foolproof to check if the beta feature is enabled for the user.

So, I guess we'll have another round of 6 patches for the extensions, and the Phan config will also need to be changed to depend on BetaFeatures for extensions that don't already depend on it.

I'm also concerned about $wgCodeEditorEnabledModes. I think that should probably $wgCodeEditorDisabledModes, since CodeEditor currently supports many more modes than CodeMirror. Or, we can adjust the logic in MediaWiki\Extension\CodeEditor::Hooks to make sure there is a value for the mode first. I think I'll do that.

Alright, so it seems we need to check the CodeMirror beta feature in CodeEditorHooks (not CodeMirrorHooks). We don't want to change the signature of the onCodeEditorGetPageLanguage method, so we need to fetch the current user from the RequestContext directly. We also need to check if the CodeMirror extension is loaded, i.e. in the event both $wgJsonConfigUseCodeEditor and $wgJsonConfigUseCodeMirror are false. For sysadmins who want neither CodeEditor or CodeMirror, but for some reason still have them installed, they will have CodeEditor forced on them. The WMF cluster will not be effected. For 3rd parties who are in this situation, I recommend setting $wgCodeEditorEnabledModes[ 'relevant-content-model' ] to false, and then you'll get the intended effect. Finally, we also want to check for the cm6enable=1 URL parameter, as per T373711#10801487.

This crosses a lot of wires and is very much not ideal… however this code would only be temporary until CodeMirror is out of beta. I don't immediately see any alternatives. I will consolidate the logic into a single static method in CodeEditor\Hooks, at least. That class already has knowledge of CodeMirror (temporarily).

Change #1170777 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/CodeEditor@master] Hooks: add public method to check conflicts with CodeMirror

https://gerrit.wikimedia.org/r/1170777

Change #1170780 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/Scribunto@master] CodeEditorHooks: add temporary check against CodeMirror beta feature

https://gerrit.wikimedia.org/r/1170780

Change #1170781 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/TemplateStyles@master] CodeEditorHooks: add temporary check against CodeMirror beta feature

https://gerrit.wikimedia.org/r/1170781

Change #1170782 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/Gadgets@master] CodeEditorHooks: add temporary check against CodeMirror beta feature

https://gerrit.wikimedia.org/r/1170782

Change #1170783 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/UploadWizard@master] CodeEditorHooks: add temporary check against CodeMirror beta feature

https://gerrit.wikimedia.org/r/1170783

Change #1170784 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/JsonConfig@master] CodeEditorHooks: add temporary check against CodeMirror beta feature

https://gerrit.wikimedia.org/r/1170784

Change #1170192 merged by jenkins-bot:

[mediawiki/extensions/Gadgets@master] CodeMirrorHooks: add integration with CodeMirror

https://gerrit.wikimedia.org/r/1170192

In T373711#11018957, @MusikAnimal wrote:

This crosses a lot of wires and is very much not ideal… however this code would only be temporary until CodeMirror is out of beta. I don't immediately see any alternatives.

I thought about it and some ideas were:

1. Disable CodeEditor and CodeMirror for everything in config, and handle the hooks in WikimediaMaintenance, loading either the CM modules or CE modules based on beta pref.
2. Same, but since WikimediaMaintenance is currently used only for maint scripts and doesn't handle any hooks, handle them directly in wmf-config instead.
3. Same, but since handling hooks in wmf-config could be error-prone due to lack of linting, testing etc, create a new CodeEditorMigration extension...

... But that would need a lot of paperwork and patience. At that point, what you are doing now seems simpler.

Change #1170777 merged by jenkins-bot:

[mediawiki/extensions/CodeEditor@master] Hooks: add public method to check conflicts with CodeMirror

https://gerrit.wikimedia.org/r/1170777

Change #1170783 merged by jenkins-bot:

[mediawiki/extensions/UploadWizard@master] CodeEditorHooks: add temporary check against CodeMirror being enabled

https://gerrit.wikimedia.org/r/1170783

Change #1170782 merged by jenkins-bot:

[mediawiki/extensions/Gadgets@master] CodeEditorHooks: add temporary check against CodeMirror being enabled

https://gerrit.wikimedia.org/r/1170782

Change #1170781 merged by jenkins-bot:

[mediawiki/extensions/TemplateStyles@master] CodeEditorHooks: add temporary check against CodeMirror being enabled

https://gerrit.wikimedia.org/r/1170781

Change #1170780 merged by jenkins-bot:

[mediawiki/extensions/Scribunto@master] CodeEditorHooks: add temporary check against CodeMirror being enabled

https://gerrit.wikimedia.org/r/1170780

Change #1170784 merged by jenkins-bot:

[mediawiki/extensions/JsonConfig@master] CodeEditorHooks: add temporary check against CodeMirror being enabled

https://gerrit.wikimedia.org/r/1170784

Thanks SD!

Scope changes (it never ends, hehe!) – Vue was added as a content model by SD this week with T340460. I started a patch and managed to get highlighting for Vue working, but it needs some cleanup. I hope to have that ready within a few days, if not I can put it up for grabs. I may need help adding a worker for the the linter as well. I figured it should be doable as there's a plugin for eslint, and we have that working already.

As for deployment, I still want to rethink how we're structuring the ResourceLoader modules, but what we have is fine for a Beta-Cluster-Infrastructure deployment (not to be confused with Beta-Feature which is in production). I want it on Beta to test the configuration, and if that all looks fine we can proceed with prod (for beta feature users). Longer-term, we do need to think about ResourceLoader optimization, as it's starting to feel like we'll be adding a lot more CM modes. CodeEditor is a good example to go by as it has been in production for many years.

Also, my apologies for the spam with all the patches! We should be creating subtasks, which I'll do moving forward. The integration patches were all basically the same thing, just repeated for each extension, so it was easier to lump them here.

In T373711#11033654, @MusikAnimal wrote:

I may need help adding a worker for the the linter as well. I figured it should be doable as there's a plugin for eslint, and we have that working already.

This plugin is ESLint 9, which means pretty much new work of browserifying. If ESLint 8 is still an option, we can simply use the existing JavaScript worker (and Stylelint worker, if desirable). My online demo shows the synergy between ESLint 8 and Stylelint.

Change #1175581 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[operations/mediawiki-config@master] beta: use CodeMirror instead of CodeEditor

https://gerrit.wikimedia.org/r/1175581

Change #1175581 merged by jenkins-bot:

[operations/mediawiki-config@master] beta: use CodeMirror instead of CodeEditor

https://gerrit.wikimedia.org/r/1175581

This is now deployed on the Beta cluster. Task followers are invited to give it a try, i.e. https://en.wikipedia.beta.wmcloud.org/wiki/Module:Message_box/ambox.css?action=edit&cm6enable=1

Note there is a bug with WikiEditor keyboard shortcuts leaking through to CodeMirror. I've filed T401271 for that.

Change #1183757 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/CodeMirror@master] Combine non-wikitext RL modules, use CodeMirrorMode interface, add Vue

https://gerrit.wikimedia.org/r/1183757

Change #1183768 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/CodeMirror@master] Add Vue mode

https://gerrit.wikimedia.org/r/1183768

Change #1183757 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/CodeMirror@master] Combine non-wikitext RL modules, and introduce CodeMirrorMode interface

https://gerrit.wikimedia.org/r/1183757

Change #1183757 merged by jenkins-bot:

[mediawiki/extensions/CodeMirror@master] Combine non-wikitext RL modules, and introduce CodeMirrorMode interface

https://gerrit.wikimedia.org/r/1183757

Change #1183768 merged by jenkins-bot:

[mediawiki/extensions/CodeMirror@master] Add Vue mode

https://gerrit.wikimedia.org/r/1183768

CodeMirror 6 + the new modes have been running well on Beta for a while. Let's get this out to all Beta feature users next week to see what folks think :)

In full transparency, I am going on sabbatical starting later this month through February (E1926). We'll have two weeks to iron out any issues that require my attention. I imagine we'll be all set by then, but if not, we have until December 23 to revert if necessary (this is just a config change).

Only users of the "Improved Syntax Highlighting" beta feature are affected, so it can always be disabled.

I'll get something written in Tech News shortly!

Is there a way to opt out from this for CodeMirror Beta users?

In T373711#11436106, @IKhitron wrote:

Is there a way to opt out from this for CodeMirror Beta users?

There's some info further up (T373711#10801487) on why it isn't particularly simple, but I'm happy to help make a gadget or something that uses the ?cm6enable=1 URL parameter (basically a hacky workaround).

However, the goal here is to eventually replace CodeEditor entirely. If there's something missing from CodeMirror that you want, just let us know :) I imagine most users won't even notice a difference, apart from perhaps the different syntax highlighting palette. You will also be able to enjoy modern linters (ESLint, Stylelint, etc.) so in contrast to CodeEditor, the info/warnings/errors will be accurate with what the wiki actually supports.

In T373711#11437119, @MusikAnimal wrote:

In T373711#11436106, @IKhitron wrote:

Is there a way to opt out from this for CodeMirror Beta users?

If there's something missing from CodeMirror that you want, just let us know :) I imagine most users won't even notice a difference, apart from perhaps the different syntax highlighting palette.

I tried CodeMirror for a couple of times, and as for now it's not comforfable for me personally to work there. And the fact that the wikicode with CodeMirror is much, and sensitively, slower does not help. Hope it will all change. And by the way, changing even one color in highlighting is a big deal, talking from a last week experience outside wiki.

In T373711#11437138, @IKhitron wrote:

I tried CodeMirror for a couple of times, and as for now it's not comforfable for me personally to work there.

If you could be more specific on what was uncomfortable about CodeMirror, we will try to improve the experience.

And the fact that the wikicode with CodeMirror is much, and sensitively, slower does not help. Hope it will all change.

There should be no discernible difference between CodeEditor and CodeMirror with respect to performance. Even wikitext should have similar speeds, but perhaps you're using the 2017 editor which is known to be slow with CodeMirror (see related discussion). At any rate, we want to hear about any all feedback, but if it's about wikitext, then let's bring it to a different task :)

And by the way, changing even one color in highlighting is a big deal, talking from a last week experience outside wiki.

It may seem drastic at first, but I promise you will get used to the different colours. We will also add themes eventually (T163533).

In T373711#11437454, @MusikAnimal wrote:

In T373711#11437138, @IKhitron wrote:

I tried CodeMirror for a couple of times, and as for now it's not comforfable for me personally to work there.

If you could be more specific on what was uncomfortable about CodeMirror, we will try to improve the experience.

It is hard for me to pinpoint that. Maybe I could try in the future, after you've asked me to, I can knowingly pay attention.

And the fact that the wikicode with CodeMirror is much, and sensitively, slower does not help. Hope it will all change.

There should be no discernible difference between CodeEditor and CodeMirror with respect to performance. Even wikitext should have similar speeds, but perhaps you're using the 2017 editor which is known to be slow with CodeMirror (see related discussion). At any rate, we want to hear about any all feedback, but if it's about wikitext, then let's bring it to a different task :)

Well, it is. The different is between "instant soft responce" to "I need to wait every single time". And no, I do not use 2017 editor.
I do not expect you will change something just because I, one person, have a problem. It is not reasonable, if the other users do not complain. This is why I didn't ask you not to do it, just wanted to know if there is a way to opt out.