Page MenuHomePhabricator

SyntaxHighlight adds trailing newline, breaking browser tests
Closed, ResolvedPublic

Description

Ever since we added Wikibase to EntitySchema’s CI (T333661), Browser tests have been failing:

00:01:57.083       + expected - actual
00:01:57.083 
00:01:57.084       -'content\t is \n\n\n here\n'
00:01:57.084       +'content\t is \n\n\n here'
00:01:57.084       
00:01:57.084       at ViewSchemaPage.assertSchemaText (webpack:///./cypress/support/pageObjects/ViewSchemaPage.js:20:40)
00:01:57.084       at Context.eval (webpack:///./cypress/e2e/viewSchema.cy.js:16:17)

This is reproducible locally iff SyntaxHighlight_GeSHi is installed; it turns out that SyntaxHighlight_GeSHi is a indirect CI dependency of Wikibase (Wikibase → ArticlePlaceholder → Scribunto → SyntaxHighlight_GeSHi), and thus it’s now loaded in EntitySchema CI as well.

We are making use of syntax highlight ourselves in EntitySchemaSlotViewRenderer. That was added for T238831: Syntax highlighting for viewing ShExC code (EntitySchema pages).

Having a look at the database, it is clear that this is only a visual change when viewing the EntitySchema content. It is not in the database and it is also not present when editing the content.

Possible ways to maybe fix this:

  1. understand how/why/where SyntaxHighlight is adding those newlines and fix it there
  2. trim those newlines in our browser tests
  3. expect those newlines in our browser tests
  4. add the newlines to our input data in tests
  5. somehow disable SyntaxHighlight in our browser tests

Event Timeline

Looks like Pygments always ignores any existing trailing newlines and adds one trailing newline:

$ printf '"abc"' | extensions/SyntaxHighlight/pygments/pygmentize -l c -f html
<div class="highlight"><pre><span></span><span class="s">&quot;abc&quot;</span><span class="w"></span>
</pre></div>
$ printf '"abc"\n' | extensions/SyntaxHighlight/pygments/pygmentize -l c -f html
<div class="highlight"><pre><span></span><span class="s">&quot;abc&quot;</span><span class="w"></span>
</pre></div>
$ printf '"abc"\n\n' | extensions/SyntaxHighlight/pygments/pygmentize -l c -f html
<div class="highlight"><pre><span></span><span class="s">&quot;abc&quot;</span><span class="w"></span>
</pre></div>
$ printf '"abc"\n\n\n' | extensions/SyntaxHighlight/pygments/pygmentize -l c -f html
<div class="highlight"><pre><span></span><span class="s">&quot;abc&quot;</span><span class="w"></span>
</pre></div>
$ extensions/SyntaxHighlight/pygments/pygmentize -V
Pygments version 2.15.0, (c) 2006-2023 by Georg Brandl, Matthäus Chajdas and contributors.

Apparently this has been reported upstream as pygments/pygments#1141, but without any reaction. I don’t think we should wait for this to be fixed in Pygments; but I’m not sure if it’s better to fix this in SyntaxHighlight, in EntitySchema’s EntitySchemaSlotViewRenderer, or in EntitySchema’s browser tests.

Oh, and apparently it’s not even possible to have leading or trailing whitespace in an EntitySchema test? In my testing, edits that only add leading or trailing spaces or newlines become null edits (no new revision in the history), and with edits that change both surrounding whitespace and other content, only the non-whitespace changes survive. We must be trimming the schema text ourselves somewhere.

I think that rules out option 4, and also means the impact of this task is limited to the browser tests, and we don’t need to e.g. ask product if it’s a problem that users don’t see any leading whitespace (because it’s not possible to have leading whitespace in the first place).

Okay, then let's trim them. Should be quick enough to do

Change 908228 had a related patch set uploaded (by Michael Große; author: Michael Große):

[mediawiki/extensions/EntitySchema@master] tests: trim schema text when asserting it in browser tests

https://gerrit.wikimedia.org/r/908228

Change 908228 merged by jenkins-bot:

[mediawiki/extensions/EntitySchema@master] tests: trim schema text when asserting it in browser tests

https://gerrit.wikimedia.org/r/908228