Background

In working on T348019: Port MediaWiki mode to CodeMirror 6 stream-parser, I have hit a wall with one particular issue where identical adjacent tokens unnecessarily have separate <span> tags. This does not visually produce any issue, but the excess markup could pose a performance problem.

I tried reproducing the issue with CodeMirror 6's "Try" feature, but in doing so I believe I have concluded this is not an issue with CM6 but with my code.

Steps to replicate the issue (include links if applicable):

• Check out r972438
• Start a new page with CM6 enabled (i.e. ?cm6enable=1)
• Add only the content https://wikimedia.org

What happens?:

The markup is as follows:

<div class="cm-line">
  <span class="cm-mw-free-extlink-protocol">https://</span>
  <span class="cm-mw-free-extlink">wikimedia.</span>
  <span class="cm-mw-free-extlink">org</span>
</div>

What should have happened instead?:

The markup should be:

<div class="cm-line">
  <span class="cm-mw-free-extlink-protocol">https://</span>
  <span class="cm-mw-free-extlink">wikimedia.org</span>
</div>

Software version

CodeMirror with r972438 checked out

Other information (browser name/version, screenshots, etc.):

The above is just one example. This happens in many cases, such as table headings:

{| class="wikitable"

produces:

<div class="cm-line">
  <span class="cm-mw-table-bracket">{|</span>
  <span class="cm-mw-table-definition">class</span>
  <span class="cm-mw-table-definition">="wikitable"</span>
</div>

There should be just one span for .cm-mw-table-definition.