Page MenuHomePhabricator

Lua's mw.text.nowiki() should escape exclamation point "!" on new line
Closed, ResolvedPublic

Description

According to the documentation, mw.text.nowiki() escapes all chars that have special meaning in wikitext. An exclamation point "!" is part of the table syntax, so it should be replaced when it starts a new line. Example wikitext:

{|
! heading
|}

Example of problem in practice:
On https://sv.wiktionary.org/wiki/Modul:grammar-table/test we have been forced to create a workaround for this issue.

Event Timeline

Change 392419 had a related patch set uploaded (by Gerrit Patch Uploader; owner: Vlakoff):
[mediawiki/extensions/Scribunto@master] Make mw.text.nowiki() escape "!" at beginning of line

https://gerrit.wikimedia.org/r/392419

Mmmm, wait. Table code would also accept such leading spaces/tabs:

{|
    ! heading
|}

New patch, that searches for \n[ \t]*!

Also added some missing tests, for \n----, __, PMID

Change 392475 had a related patch set uploaded (by Gerrit Patch Uploader; owner: Vlakoff):
[mediawiki/core@master] Make wfEscapeWikiText() escape "!" at beginning of line

https://gerrit.wikimedia.org/r/392475

As suggested on Gerrit, I added a patch that implements a similar change in core wfEscapeWikiText().

Though, it tickles me to add stuff to this function, and I'm not sure it's really useful. Could anyone provide a demo showing the need for this?

How about just escaping all ! characters? It would escape more than necessary, but the code would be much simpler (in particular the PHP one).

@Od1n , I didn't see your comment from November.

Could anyone provide a demo showing the need for this?

In this task description, I provided a use case, where we had to create a function to escape exclamation points, for text that would later be processed by mw.text.nowiki(). See our function nowiki() on https://sv.wiktionary.org/wiki/Modul:grammar-table/test.

How about just escaping all ! characters?

Fine by me.

Change 392475 abandoned by Thiemo Kreuz (WMDE):

[mediawiki/core@master] Make wfEscapeWikiText() escape "!" at beginning of line

Reason:

6 years old. Unclear why this is even needed. A ! alone is not valid syntax.

https://gerrit.wikimedia.org/r/392475

Change 392419 abandoned by Thiemo Kreuz (WMDE):

[mediawiki/extensions/Scribunto@master] Make mw.text.nowiki() escape "!" at beginning of line

Reason:

6 years old. Unclear why this is even needed. A ! alone is not valid syntax.

https://gerrit.wikimedia.org/r/392419

thiemowmde subscribed.

I'm not sure how this is a problem with the generic escaping functions. A ! character alone is not valid syntax. Escaping the surrounding table syntax is sufficient.

I'm afraid the provided example page is not helpful. What to look for? As far as I can tell this is about incomplete, so called "unbalanced" fragments of wikitext. The workaround is exactly what you did: Do the necessary escaping manually in the context where you know you are dealing with unbalanced wikitext. That's not necessarily the job of the generic escaping functions.

Change 392419 restored by C. Scott Ananian:

[mediawiki/extensions/Scribunto@master] Make mw.text.nowiki() escape "!" at beginning of line

https://gerrit.wikimedia.org/r/392419

I'm not sure how this is a problem with the generic escaping functions. A ! character alone is not valid syntax. Escaping the surrounding table syntax is sufficient.

^\s*! is, in regex terms, once you're inside the table, which you seem to recognize by adding that caveat after, which then invokes a value judgement about what is/n't sufficient.

Adding a nowiki here via mw.text.nowiki is indeed inconsistent with the expectation established by common wikitext:

image.png (134×213 px, 3 KB)
image.png (103×88 px, 946 B)

though with spaces it starts a <pre> too:

image.png (135×266 px, 3 KB)
image.png (143×166 px, 1 KB)

I don't know what part of the old parser causes this behavior so I can't say where it is an issue (IDK about Parsoid), but it's at least at the "general escaping function" level. And given the behavior is different in Lua than old parser, that indicates to me the issue is probably Lua.

Now, is this something I would be strongly motivated to correct? Not personally - I think use of mw.html in Lua is much more useful (but may not be sufficient for a use case like the above, there has been discussion about making wikitext and HTML not mix in the case of tables). But a closure of this task as Invalid is certainly incorrect.

Change 392419 merged by jenkins-bot:

[mediawiki/extensions/Scribunto@master] Synchronize mw.text.nowiki() with wfEscapeWikiText in core

https://gerrit.wikimedia.org/r/392419

Mmmm, wait. Table code would also accept such leading spaces/tabs:

{|
    ! heading
|}

Both PHP wfEscapeWikiText() and Lua mw.text.nowiki() only escape \n! (and \r!), thus the « ! » is not replaced in the above snippet.

However, the wikicode is thankfully not interpreted because we are also replacing \n<space> with \n&#32; (and \r<space> with \r&#32;).

I don't see any further actions needed. Marking the ticket as resolved.