How is it if u disable the syntaxhighlighting beta ?

@TheDJ Turning syntaxhighlighting off solved the shifting text and red underlines problems. However, the speed is still unacceptably slow.

@Bencemac Do you face similar problems when not using the 2017 Wikitext Editor?

No. For example, see the history of the page in the video. I had to use the old editor because the new cannot handle with the original 176k bytes. Firefox simply crashed down, so I temporarily turned the new one off.

In T184857#3903308, @Bencemac wrote:

No. For example, see the history of the page in the video. I had to use the old editor because the new cannot handle with the original 176k bytes. Firefox simply crashed down, so I temporarily turned the new one off.

Sorry to hear that. Are you sure that it doesn't have the same problems if syntax highlighting is disabled?

I tested with my sandbox (still 68.000 bytes) and the results show that syntax highlighting doesn't really affect the speed. Although, T180678 and T176073 still appear if it is turned on.

2017 wikitext editor: cutting 14s, pasting 8s
2017 wikitext editor with syntax highlighting: cutting 14s, pasting 9s

Old editor: instantly
Old editor with syntax highlighting: almost instantly

@Bencemac Okay. Thanks for the detailed report. I suggest you don't use 2017 editor + Syntax highlighting on very large pages for now. We'll investigate this and get back to you. Can you meanwhile tell me what device you were using for this? Which processor? And browser was Firefox, correct?

My device is a Dell Inspiron 3542 with Intel i7-4510U
Windows 10 (64bit), 1709 (build 16299.125)
Firefox 57.0.4

If you need some more information, just write me.

I had assumed that the performance and other issues with the New Editor would have been well known / well communicated within the WMF.

If the 2017-editor hadn't been effectively built in secret, a horde of editors would have told the WMF that it would be a fatally bad-idea to try to downgrade wikitext into a second-class-citizen inside of Visual Editor. As soon as the 2017 Editor prototype was available, there was immediate and strong feedback that performance and faulty previews where both fatal fatal flaws. More than a year ago an English Wikipedia RFC was initiated, which reached consensus to to block deployment of the 2017 editor as a replacement for the current wikitext editor, both on the grounds of performance and separately on the grounds of faulty previews.

On a related note, I suspect that integrating the New Editor into Flow is the cause of the (apparently new) performance and page-lockup issue I recently came across in Flow. (Not that I want the WMF wasting time trying to fix Flow, it has been overwhelmingly rejected by the community with no hope of success. However I am rather dismayed that the WMF has devoted work managing to make Flow even worse.)

I hope that one of these years the WMF and community can work together on building stuff that we actually want and need, rather than wasting so much money and hard work on projects that were known to be non-viable at or before the prototype stage.

@Alsee, I didn't build, nor do I work on the 2017 editor or Flow, but speaking from personal experience I think both of these projects are vital for bringing in and retaining new editors. I know the existing community doesn't gain anything from either of these. These are people who are well versed with nuances of wikitext and can edit talk pages in their sleep. But these very things are extremely intimidating for newcomers. Editing talk pages is a pain. Keeping track of discussions on talk pages is a pain. Understanding wikitext markup takes a long while.
You probably know that new editor enrollment/retention has been mostly falling since a while now.

I know there are performance issues and bugs. Nobody is talking about removing the existing editor or talk pages. But if we want to keep up with other websites and get more people involved in editing our projects, we have to improve the workflows around that. I cannot think of any other website which runs on a scale that ours does and has such complicated ways of editing.

I hope that one of these years the WMF and community can work together on building stuff that we actually want and need, rather than wasting so much money and hard work on projects that were known to be non-viable at or before the prototype stage.

I think the WMF very well recognizes the need for supporting the community and that is why Community Tech was created. But at the same time, WMF also has to look at the bigger picture and think about bringing in more people to our projects. I hope that the WMF and community can work together to make our projects more efficient and welcoming.

In my opinion, the 2017 wikitext editor is a very great and useful gadget. Since I use it, I usually don't have any problem with it; but when I do, it is totally useless. Editing big(ger) pages is hopeless, syntax highlighting doesn't want to work with it, many functions are missing, etc. Even thought, I always try to use it to find its bugs and report them; if we could solve them, newcomers' life would be much easier.

@Niharika, we're all supposed to be on the same team. The suggestion that the community doesn't care about new users is not merely offensive, it's dangerous.

I know everyone at the WMF has good intentions, however the road to hell is paved with good intentions. The disregard for the experience and expertise of the community, the disregard for consensus and collaborative approach, they only make it impossible for the WMF and community to work together.

If there's one thing the community is passionate about, it's defending the project. The community has well established that it will use all tools at its disposal to do so. The community looks at the big picture, and that includes bringing in productive new contributors. The community has made it abundantly clear that when a software deployment has no effect on existing users, when a deployment only affects incoming users, that the community is willing to use the sitewide javascript and all other tools at our disposal to disable, override, circumvent, disable, or fix any WMF deployment that threatens to undermine or drive off new contributors.

There is something seriously wrong with the WMF product development process and community engagement when there is mass-revolt against multiple major projects and the community is defending new users from the WMF.

And to address the Phab task, this task should be declined. The proper fix would be to either scrap the project or actually build a wikitext editor, not a 2017second-class mode inside VE that you can't deploy.

@Niharika The two above problems were solved by T180678, currently this task is just about the slow speed. I can tell you a good news, after three months it becomes faster but it is still slower than the old one. Do we need this task or should we close?

https://codemirror.net/doc/manual.html

workTime, workDelay: number
Highlighting is done by a pseudo background-thread that will work for workTime milliseconds, and then use timeout to sleep for workDelay milliseconds. The defaults are 200 and 300, you can change these options to make the highlighting more or less aggressive.

pollInterval: number
Indicates how quickly CodeMirror should poll its input textarea for changes (when focused). Most input is captured by events, but some things, like IME input on some browsers, don't generate events that allow CodeMirror to properly detect it. Thus, it polls. Default is 100 milliseconds.

Anyone tried to play with these settings ?

@TheDJ I don't think so. We had a feature where it won't render the entire page with highlighting but only render the part of the editor that is visible and that was quite efficient with large pages. However, we had to do away with that because it broke Ctrl-F as it only searched the rendered part of the article.

Nice find, TheDJ! This issue came up again at https://en.wikipedia.org/wiki/Special:Permalink/845996645#Syntax_highlighting_will_no_longer_turn_off_/_Forced_into_2017_Editor (which I assume you were following)

Maybe we could conditionally increase these settings based on the size of the page. This way it's still really responsive for smaller pages. When time allows I'll be happy to play around with it.

Editing a section of a huge page and saving the changes are very slow as well; you can try it on these pages.

The Editing team can not prioritize this task at this time.

While I haven't been able to find a solution for WE2017, I have found a trick to improve performance for WE2010 https://gerrit.wikimedia.org/r/574855

The old wikitext editor is also unbearably slow on huge pages with syntax highlighting on, so there should probably be separate tasks for CodeMirror and WTE2017?
I haven't tested, but I'm told User:Remember the dot/Syntax highlighter, which has similar functionality, performs much better in these cases.

This might be related to the bug I found and fixed in T270317#6740027.