In T331906#11091562, @Taylor wrote:

I also greatly prefer T122934 (or something similar) to this hacky workaround.

Why is this hacky? The page https://en.wiktionary.org/w/index.php?oldid=35650095&action=edit linked from there:

Thank you.

In T403113#11126664, @cscott wrote:

In T403113#11126659, @Benwing2 wrote:

For example, [[facci]] displays /fàtt͡ʃi/ instead of /'fat.t͡ʃi/.

Thank you. On https://en.wiktionary.org/wiki/facci presumably?

In T403113#11126654, @cscott wrote:

Can you provide a single page that is currently broken?

In T403113#11126641, @cscott wrote:

All I am asking for is a proper bug report here: what are the modules and pages that are affected?

In T403113#11126624, @cscott wrote:

@loaxxere We already do: mw.string.toNFD() will give you the decomposed form if that's what you need.

In T403113#11126620, @cscott wrote:

I was unable to find any breakage in {{it-pr}}, and Module:grc-translit has already been fixed. If you can provide further examples of incorrect output I am happy to work on fixes.

In T403113#11126594, @cscott wrote:

Well, scribunto already provides mw.ustring.toNFD so for cases where non-NFC text is needed, it is available. Language::uc() and friends are primarily meant for manipulating wikitext and titles, and NFC output in those cases seems appropriate.

In T403113#11126392, @cscott wrote:

Wikitext content is supposed to always be in NFC form. Can you describe further what you are trying to do? It sounds to me like the Greek Transliteration module is perhaps broken? What else is relying on non-normalized text?

I wrote this code. I give a detailed explanation of the issue below, but the tl;dr is that get_current_L2 relies on a nasty kludge to get the current section number (the number used to determine the section when you edit a section), and this kludge seems to be incompatible with Parsoid.

In T363118#9805550, @Tchanders wrote:

Thanks @Novem_Linguae. To anyone else reading this, see also T355392: Confirm and update what information IPInfo users should be able to see.

@Zache Thanks for the additional info. The "not available" error would be a coverage issue, meaning that the IP isn't in the Spur database. I've made a related comment here: T355392#9805549.

In T365065#9805459, @cscott wrote:

See T84937#957838 for justification when support for nbsp was added. The new entities added here aren't likely to be generated by Visual Editor so it's not entirely clear why we need to support them; wikitext is not a superset of HTML5.

But in any case at a bare minimum you'd need a patch against Parsoid as well as the core parser, and parser tests for the new functionality.

The main issue that I see with both this and T331906 is that neither has a good way to handle (a) headings parsed by the preprocessor which fail on expansion (e.g. a heading containing a template which expands into multiline text), or (b) headings which aren't parsed as such by the preprocessor, but which are created via expansion (e.g. headings in template outputs).

In T122934#9196348, @cscott wrote:

I'm not in favor of T331906 at all. This task (or something like it) is a much better solution.

We do have <section> tags in Parsoid read views. I'm not entirely convinced that hanging the data on the section is the right approach. My personal feeling is that variable declarations in a patch should be scoped and that heredoc quoting (T114432) will greatly help with the visual appearance of this. I think something like this is all that's needed:

{{#set|lang=en|<<<

...the rest of the content here...

>>>}}}

Somewhat related is the idea of "page context data" due to SMW, see https://en.wikipedia.org/w/index.php?title=File:SMWCon_2022_-_Updates_from_the_Wikitext_Parsing_World_01.pdf&page=14 for some details. I don't know if this "page variable" data should be folded into the page context data or not, but my intuition is that they are aspects of the same thing. Unresolved is how this data is passed down into template/subpage invocations. As discussed some in T114454 (referenced above in T122934#1918310), a handlebars style evaluation context could have additional scopes pushed onto it via {{#set}} or altered using {{#each}} or {{#with}} and then can be explicitly (not implicitly) passed into subpages or modules (see the "block helpers" section of the handlebars documentation).

I think it is important that templates *not* have implicit access to the evaluation context, because once you do that then tracking dependencies between page fragments becomes very complicated, and you can't cache and reuse a page fragment without the implicit evaluation context being identical, which then means it needs to be serialized and used as part of the lookup key, etc.

In T357199#9574226, @Od1n wrote:

While doing some random test, I noticed the 4 assignments local getmetatable = getmetatable, etc. actually make the code slower.
What makes the code slower is not the assignment, but the access to the local variable instead of the global function. It's as if the Lua engine had some internal optimization for these global accesses.

Another idea I had for this is to make most of the data in mw.site available via a metatable, in the same way mw.loadData is. Removing mw.site and package.loaded["mw.site"] from _G speeds up mw.clone(_G) by about 4.5 times, which is a major reduction, and there's no reason for anyone to be writing to those tables anyway.

In T165935#9702723, @NickK wrote:

It does seem to happen again, e.g. https://en.wiktionary.org/wiki/opistho- . It's not a particularly problematic page though, still the whole page is one big 'not enough memory' error

Also see https://en.wiktionary.org/wiki/Wiktionary:Grease_pit/2024/April#Lua_error

In T357199#9574367, @alex-mashin wrote:

Is there a reason not to submit a patch, summarising the improvements above, immediately?

Here's a version that's about 15% faster than the original:

• It avoids generating a new closure every time the main function is called.
• tableRefs isn't an upvalue, so access is faster.
• Avoids any global variables.

I should have mentioned why I chose for key, elt in next, val do: it's because it avoids any __pairs metamethod, which would prevent a true clone of the table being created. This wouldn't work with data loaded via mw.loadData, but (a) mw.clone already throws an error if you pass data loaded via mw.loadData into it, and (b) even if it didn't, it defeats the purpose of mw.loadData to make a local clone of the data, so you may as well just load it via require in the first place.

From my tests, the speed improves by about 10% if the check for non-tables and already-seen values is in the for-loop, rather than at the start of the recursive call. I've also minimised the number of table accesses.

Strongly support this. At the English Wiktionary, we use a recursive backtracking parser to iterate over templates, since we have to do a lot of data scraping due to a large amount of info being spread across pages and to ensure we remain accessible (instead of shoving everything into intimidating data tables). The current performance is good, but it would be even better with coroutines, since there'd be no need to build the whole node tree of templates before traversing it. This is an issue on very large pages, where any breaks necessarily have to happen in the second pass.

Just to reiterate Erutuon's point above, that approximately 1.6% of all mainspace pages on the English Wiktionary- that's a lot! Obviously we can adjust the modules to account for this, but it's a situation that was difficult to spot, and causes problems in sorting that are difficult to debug since it's so unintuitive. It really should be being treated as a bug, not a feature request.

In T354139#9469497, @cscott wrote:

I'm not sure I agree entirely with the bug summary. I suspect that the "right" wikitext for this case would use an ampersand-escape for the category sort key:

[[Category:&amp&foo&&]]

which is the standard way to flag to the parser that this is not intended to be wikitext markup, and avoids having /some/ [[ link captions parsed as wikitext and /some/ [[ link captions parsed as raw text. That would also ensure that the name of the category displays properly on the category list page.

But in order for that to work, the category code should probably html-decode the sort key before doing the actual sort. Not sure whether that is done or not; if not then that's a legit bug/feature request that could be addressed.

This has been pending for three months now with no action. Could we please get an update?

Thanks for the quick turnaround.

In T165935#9276399, @TheresNoTime wrote:

Just FYI, although https://en.wiktionary.org/wiki/a now loads without errors, the following (expected) log entry is generated: Parsing 'a' was slow, took 30.63 seconds

Why has this been closed? This ticket is for Wiktionary, not Wikiversity.

In T344001#9084645, @Legoktm wrote:

of a Lua wikitext parser that I've been writing

why? https://bash.toolforge.org/quip/AU7VVyPV6snAnmqnK_0N is relevant here.

@Pcoombe [[en:wikt:summary]], [[en:wikt:heading]] and [[en:wikt:actions]] may also be exhibiting unintended behaviour, in that case. There are a few other classes in that format, but the titles are implausible for Wiktionary.

Hi @Aklapper - I've updated the initial report with the browsers I've tested it on.

Just to point out that this is a common practice on English Wiktionary when striking out a whole section and works without issue in other skins, so regardless of whether it's bad HTML from a technical perspective, it does need to be corrected as some users will continue to do this if they aren't using Vector 2022.