brianna.laugher wrote:

Can't be too hard to fix... can it? :)

robchur wrote:

*** Bug 6222 has been marked as a duplicate of this bug. ***

goldom wrote:

sorry bout 6222, I did a search for <gallery> first but that wasn't in this
one's name.

• Bug 6157 has been marked as a duplicate of this bug. ***

Fixed in SVN.

brianna.laugher wrote:

So... when should this take effect? (
http://en.wikibooks.org/wiki/User:Pfctdayelise has a gallery with these pipes)

robchur wrote:

When it's been reviewed and upon the next synchronisation, like all other fixes
and additions.

Patch is bogus, does not fix reported cases.

hirsch.heisseich wrote:

It seems to work for me. Doesn't it?
Greetings

ayg wrote:

*** Bug 8125 has been marked as a duplicate of this bug. ***

ayg wrote:

*** Bug 6814 has been marked as a duplicate of this bug. ***

ayg wrote:

*** Bug 8929 has been marked as a duplicate of this bug. ***

• Bug 10927 has been marked as a duplicate of this bug. ***

• Bug 12016 has been marked as a duplicate of this bug. ***

• Bug 13072 has been marked as a duplicate of this bug. ***

• Bug 13959 has been marked as a duplicate of this bug. ***

• Bug 14007 has been marked as a duplicate of this bug. ***

nicdumz wrote:

Seems that it is broken again :)

http://fr.wikipedia.org/w/index.php?title=Alaska&diff=prev&oldid=29263402 references is a good example of [[title (disambig)|]] not showing properly

• Bug 13120 has been marked as a duplicate of this bug. ***

cuerden wrote:

So, with all the closing down of other bugs as duplicates of this... when is someone going to make an attempt to fix it?

• Bug 14550 has been marked as a duplicate of this bug. ***

*** Bug 15547 has been marked as a duplicate of this bug. ***

sirabder87 wrote:

Still not working subst within reference (e.g.: http://it.wikipedia.org/w/index.php?title=Hardcore_punk&diff=18368819&oldid=18364416 )

wiki.bugzilla wrote:

*** Bug 16183 has been marked as a duplicate of this bug. ***

wiki.bugzilla wrote:

*** Bug 15467 has been marked as a duplicate of this bug. ***

wiki.bugzilla wrote:

*** Bug 16904 has been marked as a duplicate of this bug. ***

bluehairedlawyer wrote:

This one's been around for quite a while...

The problem here is that MediaWiki treats all extension tags as if they don't contain wikitext, so extensions that do contain wikitext, such as Cite, have to call the parser themselves. And in so doing miss out on the pre-save transformations.

Ideally we should be able to set extension hooks so that their wikitext output gets parser along with everything else.

herd wrote:

*** Bug 18064 has been marked as a duplicate of this bug. ***

#tag allows you to use that trick in refs and other tags

see [[mw:Help:Magic words#Miscellaneous]]

I have moved the severity up to normal. This bug causes problems with using apparently standard wiki-syntax in certain places where it could be expected to be usable. As such it requires special case handling or adoption of sub-optimal approaches to certain tasks. I would have thought that extensions could be flagged as requiring or not requiring pre-save processing.

bluehairedlawyer wrote:

a patch to allow extensions to output wikitext

This patch allows extensions to specify that they contain and output wikitext. The result being that such extensions don't have to invoke the parser themselves and the pre-save transformation work as they should do.

Unfortunately this doesn't work for either gallery or imagemap tags as they don't contain pure wikitext.

attachment diff.txt ignored as obsolete

ayg wrote:

*** Bug 18825 has been marked as a duplicate of this bug. ***

cbm.wikipedia wrote:

Note that the syntax {{#tag:ref|{{subst:today}}}} works as expected.

• Bug 20359 has been marked as a duplicate of this bug. ***

bluehairedlawyer wrote:

CBM + Umherirrender:

The whole point here is that we're trying to keep it as simple as possible. There's no point trying to tell people about #ref when a fair number of people over at enwiki can't figure out relative links. The best thing to do is to make work what most editors think should work. This bug is now 4 years old and has 19 deplicates. That should tell you enough.

• Bug 13141 has been marked as a duplicate of this bug. ***

• Bug 22258 has been marked as a duplicate of this bug. ***

conrad.irwin wrote:

The pipe-trick part of this is now resolved. subst: can be fixed by using safesubst: instead, so it's just signatures left.

mediawiki wrote:

For information only, this bug and the use of {{subst:...}} inside a <ref> is mentioned in [[fr:Wikipédia:Guilde_des_Guides/semaine_11_2010]].

Just a note: Conrad’s changes mentioned in comment 38 had been reverted in r62689.

" subst: can be fixed by using safesubst: "

So we tell all editors everywhere to use safesubst: instead of subst: ? Or what? I don't see this as a fix, more a work-around - but then I haven't experimented with safesubst: so perhaps I'm missing the point.

• Bug 24131 has been marked as a duplicate of this bug. ***

• Bug 24160 has been marked as a duplicate of this bug. ***

nzmoihue wrote:

no, safesubst can not
testcase:
<ref>{{safesubst:formatnum:12321123}}</ref><references/>

• Bug 24446 has been marked as a duplicate of this bug. ***

If the bug is not present when using {{#ref instead of <ref> then I asume the bug isn't as big as we might think ? They should both render the same, right ?

ayg wrote:

They'll render the same if passed the same input. <ref> is passed input without the PST being run, {{#tag:ref}} is passed the input after the PST is run on it. Something like that. I'm pretty sure some people know exactly what the bug is and have some idea how to fix it, but I'm not one of them.

(In reply to comment #47)

They'll render the same if passed the same input. <ref> is passed input
without the PST being run, {{#tag:ref}} is passed the input after the PST is
run on it. Something like that.

That's exactly right: PST runs on {{stuff in braces}} but not on <tags>.

I'm pretty sure some people know exactly what
the bug is and have some idea how to fix it, but I'm not one of them.

What I think happens is <tags> get armored before PST; <nowiki> needs this treatment, but for other tags it causes this bug.

• Bug 26859 has been marked as a duplicate of this bug. ***

The simple way to fix the original problem is to accept the pipe trick natively. So even if they are not done in the PST, they render as if they were. And just ignore things like signatures/subst inside references.

The other option is to start using a transparent hook.

happy.melon.wiki wrote:

*** Bug 28680 has been marked as a duplicate of this bug. ***

(In reply to comment #50)

The simple way to fix the original problem is to accept the pipe trick
natively. So even if they are not done in the PST, they render as if they were.
And just ignore things like signatures/subst inside references.

The other option is to start using a transparent hook.

I think having subst inside <ref> is something desirable and substs shouldn't be ignored inside references. Templates like {{cite web}} and others used on Wikipedia are an example; if we could {{subst:DATE}} in them, we didn't have to type in the current date in them.

You can use things like {{subst:DATE}}. But then the {{cite web}} template itself where you include it needs to be substed as well.

(In reply to comment #53)

You can use things like {{subst:DATE}}. But then the {{cite web}} template
itself where you include it needs to be substed as well.

You mean something like <ref>{{subst:cite web | … | accessdate = {{subst:DATE}} }}</ref>? That doesn't work.

Try using {{#tag:ref|{{subst:cite web | … | accessdate = {{subst:DATE}} }} }}

That's great, but subst'ing {{Cite web}} (etc.) would be a disaster. Simply for "Subst:" this is a basic problem which people have had to code round in Javascript and bot code. Leaving this bug unassigned means increasing amounts of resource are used to code for these special cases (which, of course rely on the underlying templates (where appropriate) not changing) as more tools are developed to work on Wikitext.

For example SmackBot (now Helpful Pixe Bot) used to use "Subst:CURRENTMONTHNAME" to date tags. Mainly because of this bug I had to write code to regenerate the XML for AWB to use the explicit monthname, a side effect being that all changes to the XML had to be back-ported to the component model. Moreover if I didn't regenerate the XML the wrong month was used.

Now AWB is no longer used for this I use the current system ides of the GMT month, which is pretty good, but still creates an unneeded dependency on my system clock.

Similarly there are probably workarounds for the other extensions which are not processed but the idea of having a mark-up language that works one way on parts of the page and another on other parts, except through design, is a real bad one.

Although it doesn't currently cause _me_ problems -

I'd like to see this prioritized.

• Bug 29775 has been marked as a duplicate of this bug. ***

*** Bug 30135 has been marked as a duplicate of this bug. ***

Can I give another example of what I think is the same issue, the inability to perform the pipe trick within {{#ref:}}

Example that does not work (at enWS)

{{subst:#tag:ref|[[Author:Charles Darwin|]]}}

should expand to give <ref>[[Author:Charles Darwin|Charles Darwin]]</ref> though it just replicates <ref>[[Author:Charles Darwin|]]</ref>. I am not seeing other issues, though I am willing to undertake trials if there are tests that are desire.

(In reply to comment #59)

Can I give another example of what I think is the same issue, the inability to
perform the pipe trick within {{#ref:}}

Example that does not work (at enWS)

{{subst:#tag:ref|[[Author:Charles Darwin|]]}}

That will never work— subst only applies to templates and #tag is a magic word. What you are trying is equivalent to <subst:ref> which will not work.

What will never work? The subst: expansion of the tag? Seems to work perfectly for me and has been for a while.

My issue is that the piping trick[1] stops working when it is inside either the <ref> or {{#tag:ref|...}}

[1] http://en.wikipedia.org/wiki/Help:Pipe_trick

I just noticed that subst isn't working inside of <ref> tags:
https://pt.wikipedia.org/w/index.php?diff=prev&oldid=28035465

I believe this is the same bug.

Pipe trick doesn't work with Translate extension popups, I guess it's this bug again. https://translatewiki.net/w/i.php?title=MediaWiki:Bw-desc-freecol/it&diff=prev&oldid=3550365

(In reply to comment #63)

Pipe trick doesn't work with Translate extension popups, I guess it's this bug
again.
https://translatewiki.net/w/i.php?title=MediaWiki:Bw-desc-freecol/it&diff=prev&oldid=3550365

That does not sounds like this, because it is not wrapping within a extension tag. Edit over GUI works for that title, this sounds like a problem with the edit api. You should fill a new bug for that.

Nx.devnull wrote:

Patch to allow extensions to specify a function that is executed during preSaveTransform

This patch introduces a mechanism similar to the tag hooks, the only difference is that the callback function gets called when the output type is set to OT_WIKI (which normally means during preSaveTransform), and that the enclosing tags with the parameters are not removed.

An extension can register a callback and then do pre-save transformation on its content - this is the best solution in my opinion, since only the extension knows which parts of the content are supposed to be transformed, e.g. in the case of the gallery tag only the description part.

This is a work in progress/proof of concept, in particular the galleryPST function calls parser->preSaveTransform, which might cause problems if it breaks the parser state. This will probably need something re-entrant like recursiveTagParse.

All parser tests passed.

Attached: tagHookPST.patch

Thanks for this patch! We've been in a code slush (http://lists.wikimedia.org/pipermail/wikitech-l/2012-January/057313.html) for a
few weeks so we're just getting around to looking at these.

We're also doing a lot of parser work, so I'm not sure how relevant this is,
but I'll ask them to take a look, but I'll still get someone to look.

wicke wrote:

(In reply to comment #65)

An extension can register a callback and then do pre-save transformation on its
content - this is the best solution in my opinion, since only the extension
knows which parts of the content are supposed to be transformed, e.g. in the
case of the gallery tag only the description part.

I am not too enthusiastic about encouraging extensions to do their own arbitrary pre-save transform processing. In the longer run, general support for extensions working with parsed (and pre-save transformed) wikitext / HTML DOM input could be cleaner and should offer better opportunities for integration with a visual editor. I am working on something like that in the Parsoid parser (http://mediawiki.org/wiki/Parsoid). There is also a simple and undocumented method for something vaguely like this in the PHP parser (the setTransparentTagHook method).

However, I do not have a good solution for partly-parsed and -transformed extensions like gallery. Will give this some thought, and ask others about their ideas.

• Bug 34999 has been marked as a duplicate of this bug. ***

(In reply to comment #67)

I am not too enthusiastic about encouraging extensions to do their own
arbitrary pre-save transform processing.

lowering priority

(In reply to comment #69)

(In reply to comment #67)
> I am not too enthusiastic about encouraging extensions to do their own
> arbitrary pre-save transform processing.

lowering priority

I am not sure how to equate the "lowering the priority" for a significant issue that doesn't follow normal wiki behaviour? We get numerous responses akin to "I don't like it" however no solutions.

Cite must be one of the most commonly used extensions in wikipedia space, and one would think that it is used multiple times per page. We have a fairly common practice with the renowned pipe trick, and now at comment #69, you are saying that we are "lowering the priority". How about we consider the basic users and solve some of the base underlying issues that bug users every day, or isn't that sexy enough to gather attention? For some of us this is a large annoyance, where we have to create work arounds, by scripting buttons, or doing significant copy and paste.

wicke wrote:

@Billinghurst: Cite can be covered by something like bluehairedlawyer's patch (attachment 5952 / comment 31), as its entire content is parsed as wikitext.

• Bug 35954 has been marked as a duplicate of this bug. ***

lambdav wrote:

More than 6 years after this bug as been reported, the bug is still there !
How many time should we wait for correction ?
Priority must not be Low !

lambdav wrote:

Priority = High because blocking another High priority bug.

Nx.devnull wrote:

(In reply to comment #67)

(In reply to comment #65)

> An extension can register a callback and then do pre-save transformation on its
> content - this is the best solution in my opinion, since only the extension
> knows which parts of the content are supposed to be transformed, e.g. in the
> case of the gallery tag only the description part.

I am not too enthusiastic about encouraging extensions to do their own
arbitrary pre-save transform processing. In the longer run, general support for
extensions working with parsed (and pre-save transformed) wikitext / HTML DOM
input could be cleaner and should offer better opportunities for integration
with a visual editor. I am working on something like that in the Parsoid parser
(http://mediawiki.org/wiki/Parsoid). There is also a simple and undocumented
method for something vaguely like this in the PHP parser (the
setTransparentTagHook method).

However, I do not have a good solution for partly-parsed and -transformed
extensions like gallery. Will give this some thought, and ask others about
their ideas.

If the problem is that extensions could do arbitrary processing instead of the standard PST, then the only solution I can think of is to allow extension to somehow specify which part of their input should be transformed, and then let the parser do the processing. Implementing that would probably be much harder and more cumbersome.

The only thing that needs to be done on my patch is that I'm not sure the extension calling parser->preSaveTransform is 100% safe. I can give it a go if you want, but I don't want to waste my time if the patch will be rejected because my approach (letting the extension do the PST) is not the correct one.

New parsers are nice and all, but they do not fix the problem that we have right now, and they probably never will if you want to keep the current gallery syntax.

bluehairedlawyer wrote:

Actually David's wrong. This bug is seven not six years old. If its not fixed soon we'll have to figure out who's going to pay for it to go to college...

What would be wrong with running the all the contents of a <gallery> through preSaveTransform? I realise it's not wikitext but then preSaveTransform isn't the full parser? What particular pre-save transforms would corrupt the gallery syntax?

Nx.devnull wrote:

(In reply to comment #76)

Actually David's wrong. This bug is seven not six years old. If its not fixed
soon we'll have to figure out who's going to pay for it to go to college...

What would be wrong with running the all the contents of a <gallery> through
preSaveTransform? I realise it's not wikitext but then preSaveTransform isn't
the full parser? What particular pre-save transforms would corrupt the gallery
syntax?

I don't know off the top of my head what characters are allowed in filenames (in particular, is [, { or ~ allowed), but in theory:

<gallery>
File:Oh noez [[This is not a link.png|]] yeah, I know, this is a contrived example
</gallery>

There's also replacing tildes, substing and the reverse pipe trick. Replacing tildes would be the most realistic problem (if ~ is allowed in filenames)

[ { are both disallowed in any page name (which includes images). Three or more ~'s in a row is also forbidden in page titles. So you don't have to worry about that.

Personally I favour an approach of adding an interface for adding parserHooks that allows specifying options (such as if the parser hook should allow PST).

Actually David's wrong. This bug is seven not six years old. If its not fixed
soon we'll have to figure out who's going to pay for it to go to college...

That's still a good 11 years away. We're cross that bridge when we come to it.

bluehairedlawyer wrote:

Some parser tests

Attached: pipeTrickTests.txt

bluehairedlawyer wrote:

Allow extensions to specify tag contents can be passed through preSaveTransform without being escaped

This is an alternative to Nikola's patch. It allows extensions and internal setHook functions to specify that there contents can be run through the preSaveTransform function without being escaped.

It passes all the current parser tests and a few others I've managed to throw at it.

Attached: preSaveTransform-diff.txt

sumanah wrote:

bluehairedlawyer, could you put this patch in Gerrit?

bluehairedlawyer wrote:

Dan Collins has already done this (which is just as well as I haven't figured out how to do it yet). Here's the url:

https://gerrit.wikimedia.org/r/#/c/8784/

That was a smart way to solve it.

Commented some concerns on gerrit.

patch moved to gerrit, removing patch need review.

Dan Collins has two patches, not one:
https://gerrit.wikimedia.org/r/#/c/8784/ (previously mentioned)
and
https://gerrit.wikimedia.org/r/#/c/8785/

• Bug 42078 has been marked as a duplicate of this bug. ***

Somebdy should really give those patches some love, as apparently the original committer has disappeared.

Gabriel: As you assigned this report to yourself a year ago, do you plean to rework the two patches?

Gabriel: As you assigned this report to yourself two years ago, do you plan to rework the two patches? Or are they "somehow" obsoleted in the meantime?

Change 8784 abandoned by Siebrand:
(Bug 2700) Pre-save transform inside extension hooks

Reason:
I'm abandoning this patch set as it's been open for a long time without any outlook at the open issues being resolved.

If the submitter or anyone else would like to work on it again, this patch set can be re-activated by clicking "Restore Change". Please only do this if you are actually going to work on it immediately.

https://gerrit.wikimedia.org/r/8784

• Bug 72079 has been marked as a duplicate of this bug. ***