How is it if u disable the syntaxhighlighting beta ?

@TheDJ Turning syntaxhighlighting off solved the shifting text and red underlines problems. However, the speed is still unacceptably slow.

@Bencemac Do you face similar problems when not using the 2017 Wikitext Editor?

No. For example, see the history of the page in the video. I had to use the old editor because the new cannot handle with the original 176k bytes. Firefox simply crashed down, so I temporarily turned the new one off.

In T184857#3903308, @Bencemac wrote:

No. For example, see the history of the page in the video. I had to use the old editor because the new cannot handle with the original 176k bytes. Firefox simply crashed down, so I temporarily turned the new one off.

Sorry to hear that. Are you sure that it doesn't have the same problems if syntax highlighting is disabled?

I tested with my sandbox (still 68.000 bytes) and the results show that syntax highlighting doesn't really affect the speed. Although, T180678 and T176073 still appear if it is turned on.

2017 wikitext editor: cutting 14s, pasting 8s
2017 wikitext editor with syntax highlighting: cutting 14s, pasting 9s

Old editor: instantly
Old editor with syntax highlighting: almost instantly

@Bencemac Okay. Thanks for the detailed report. I suggest you don't use 2017 editor + Syntax highlighting on very large pages for now. We'll investigate this and get back to you. Can you meanwhile tell me what device you were using for this? Which processor? And browser was Firefox, correct?

My device is a Dell Inspiron 3542 with Intel i7-4510U
Windows 10 (64bit), 1709 (build 16299.125)
Firefox 57.0.4

If you need some more information, just write me.

I had assumed that the performance and other issues with the New Editor would have been well known / well communicated within the WMF.

If the 2017-editor hadn't been effectively built in secret, a horde of editors would have told the WMF that it would be a fatally bad-idea to try to downgrade wikitext into a second-class-citizen inside of Visual Editor. As soon as the 2017 Editor prototype was available, there was immediate and strong feedback that performance and faulty previews where both fatal fatal flaws. More than a year ago an English Wikipedia RFC was initiated, which reached consensus to to block deployment of the 2017 editor as a replacement for the current wikitext editor, both on the grounds of performance and separately on the grounds of faulty previews.

On a related note, I suspect that integrating the New Editor into Flow is the cause of the (apparently new) performance and page-lockup issue I recently came across in Flow. (Not that I want the WMF wasting time trying to fix Flow, it has been overwhelmingly rejected by the community with no hope of success. However I am rather dismayed that the WMF has devoted work managing to make Flow even worse.)

I hope that one of these years the WMF and community can work together on building stuff that we actually want and need, rather than wasting so much money and hard work on projects that were known to be non-viable at or before the prototype stage.

@Alsee, I didn't build, nor do I work on the 2017 editor or Flow, but speaking from personal experience I think both of these projects are vital for bringing in and retaining new editors. I know the existing community doesn't gain anything from either of these. These are people who are well versed with nuances of wikitext and can edit talk pages in their sleep. But these very things are extremely intimidating for newcomers. Editing talk pages is a pain. Keeping track of discussions on talk pages is a pain. Understanding wikitext markup takes a long while.
You probably know that new editor enrollment/retention has been mostly falling since a while now.

I know there are performance issues and bugs. Nobody is talking about removing the existing editor or talk pages. But if we want to keep up with other websites and get more people involved in editing our projects, we have to improve the workflows around that. I cannot think of any other website which runs on a scale that ours does and has such complicated ways of editing.

I hope that one of these years the WMF and community can work together on building stuff that we actually want and need, rather than wasting so much money and hard work on projects that were known to be non-viable at or before the prototype stage.

I think the WMF very well recognizes the need for supporting the community and that is why Community Tech was created. But at the same time, WMF also has to look at the bigger picture and think about bringing in more people to our projects. I hope that the WMF and community can work together to make our projects more efficient and welcoming.

In my opinion, the 2017 wikitext editor is a very great and useful gadget. Since I use it, I usually don't have any problem with it; but when I do, it is totally useless. Editing big(ger) pages is hopeless, syntax highlighting doesn't want to work with it, many functions are missing, etc. Even thought, I always try to use it to find its bugs and report them; if we could solve them, newcomers' life would be much easier.

@Niharika, we're all supposed to be on the same team. The suggestion that the community doesn't care about new users is not merely offensive, it's dangerous.

I know everyone at the WMF has good intentions, however the road to hell is paved with good intentions. The disregard for the experience and expertise of the community, the disregard for consensus and collaborative approach, they only make it impossible for the WMF and community to work together.

If there's one thing the community is passionate about, it's defending the project. The community has well established that it will use all tools at its disposal to do so. The community looks at the big picture, and that includes bringing in productive new contributors. The community has made it abundantly clear that when a software deployment has no effect on existing users, when a deployment only affects incoming users, that the community is willing to use the sitewide javascript and all other tools at our disposal to disable, override, circumvent, disable, or fix any WMF deployment that threatens to undermine or drive off new contributors.

There is something seriously wrong with the WMF product development process and community engagement when there is mass-revolt against multiple major projects and the community is defending new users from the WMF.

And to address the Phab task, this task should be declined. The proper fix would be to either scrap the project or actually build a wikitext editor, not a 2017second-class mode inside VE that you can't deploy.

@Niharika The two above problems were solved by T180678, currently this task is just about the slow speed. I can tell you a good news, after three months it becomes faster but it is still slower than the old one. Do we need this task or should we close?

https://codemirror.net/doc/manual.html

workTime, workDelay: number
Highlighting is done by a pseudo background-thread that will work for workTime milliseconds, and then use timeout to sleep for workDelay milliseconds. The defaults are 200 and 300, you can change these options to make the highlighting more or less aggressive.

pollInterval: number
Indicates how quickly CodeMirror should poll its input textarea for changes (when focused). Most input is captured by events, but some things, like IME input on some browsers, don't generate events that allow CodeMirror to properly detect it. Thus, it polls. Default is 100 milliseconds.

Anyone tried to play with these settings ?

@TheDJ I don't think so. We had a feature where it won't render the entire page with highlighting but only render the part of the editor that is visible and that was quite efficient with large pages. However, we had to do away with that because it broke Ctrl-F as it only searched the rendered part of the article.

Nice find, TheDJ! This issue came up again at https://en.wikipedia.org/wiki/Special:Permalink/845996645#Syntax_highlighting_will_no_longer_turn_off_/_Forced_into_2017_Editor (which I assume you were following)

Maybe we could conditionally increase these settings based on the size of the page. This way it's still really responsive for smaller pages. When time allows I'll be happy to play around with it.

Editing a section of a huge page and saving the changes are very slow as well; you can try it on these pages.

The Editing team can not prioritize this task at this time.

While I haven't been able to find a solution for WE2017, I have found a trick to improve performance for WE2010 https://gerrit.wikimedia.org/r/574855

The old wikitext editor is also unbearably slow on huge pages with syntax highlighting on, so there should probably be separate tasks for CodeMirror and WTE2017?
I haven't tested, but I'm told User:Remember the dot/Syntax highlighter, which has similar functionality, performs much better in these cases.

This might be related to the bug I found and fixed in T270317#6740027.

Noting this is still an issue for the 2017 editor + CodeMirror 6, but reports from Fandom (who has had it deployed for some time) is that it at least performs better than CodeMirror 5. All other editors should be fine once CodeMirror 6 is fully rolled out.

On some pages, the browser tab can sometimes completely crash (example). It may have been like this before too, and it is only for very large pages, but still… this is pretty bad!

I've been doing some thinking. The core issue seems to be that the 2017 editor lacks a finite viewport. When CodeMirror is enabled, we overlay the VE surface on top of CodeMirror's, so it almost begs the question if VE should even be responsible for presentation in this scenario? Maybe instead, the user should interact with the CodeMirror contenteditable, and transactions are bubbled to the VE surface, rather than the opposite. That way we can have a finite viewport (great performance), and take advantage of CodeMirror features like template folding, highlighting of invisible control characters, respecting the user's editor font preference, and more. Currently we have to forcibly disable certain CodeMirror features in the 2017 editor.

It'd be a lot of effort to rework the 2017 editor in this way, but it seems in the long run that's probably our best bet if we are to ever close this task?

On some pages, VE + CM6 (example) stops highlighting after a few lines, while VE + CM5 (example) looks good.

CM6 parser's performance will be substantially improved by r1031962. For example, it can shorten the parsing time by 92% on Node.js (2.642 s -> 0.213 s) using the example EN Wikipedia page I mentioned.

Patch demo

Change #1253774 had a related patch set uploaded (by Bhsd; author: Bhsd):

[mediawiki/extensions/CodeMirror@master] CodeMirrorMediaWiki: performance improvement

https://gerrit.wikimedia.org/r/1253774

Change #1253774 merged by jenkins-bot:

[mediawiki/extensions/CodeMirror@master] CodeMirrorMediaWiki: performance improvement

https://gerrit.wikimedia.org/r/1253774

I'm fairly sure the problem is we're using forcing an infinite viewport in CodeMirror

I have confirmed this, to no surprise. While a fix seems attainable, it's looking like it will be a considerable amount of work. Adding a finite viewport to MobileFrontend was much easier as the editor takes 100% of the width and height, which is not the case with desktop article VE. Especially with all the different skins, we might be fighting an uphill battle here. There are also changes to the UI that may need design treatment or approval.

If worst comes to worst, I propose we move the CM5 modules to a new .v5 namespace, and mark them deprecated, and let the 2017 editor keep using CM5 until we get it sorted

I am now proposing something else: We automatically disable CM in the 2017 editor if the document is over a certain size. Things only really break or are sluggish in extreme edge cases, after all. I can do some experiments to figure out an appropriate threshold, and will report back here with examples.

Also noting that while @Bhsd's recent patch did bring a great performance improvement when parsing very large articles (yay!), the 2017 editor on Firefox is still much too slow or even non-functional for articles like 2024 United States presidential election.

I'm convinced that in the long run, we're better off either completely redoing the CM + 2017 editor integration from the ground up, or just accepting that VE + CM is not something we can support. We've been hacking away at this integration for years, and we have never been able to completely fix or even mitigate issues with misalignment and performance. Though I've been discouraged from trying, I believe it should be possible to keep the VE toolbar as it is, and let the surface itself be only CodeMirror. CodeMirror has contextual knowledge of where the cursor is, after all, so seemingly it could emit events to the toolbar, almost sort of mimicking a VE surface (from the toolbar's perspective). I'll save exploration of this for the Wikimedia-Hackathon-2026.

Change #1268195 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/CodeMirror@master] ve.ui.CodeMirrorTool.v6: automatically disable on large documents

https://gerrit.wikimedia.org/r/1268195

Change #1268195 merged by jenkins-bot:

[mediawiki/extensions/CodeMirror@master] ve.ui.CodeMirrorTool.v6: automatically disable on large documents

https://gerrit.wikimedia.org/r/1268195