Pre-save transform skips extensions using wikitext (gallery, references, footnotes, Cite, status indicators, pipe trick, subst, signatures)
Open, HighPublic

Subscribers
Tokens
"Burninate" token, awarded by Daimona."Love" token, awarded by Liuxinyu970226."Burninate" token, awarded by Pols12."Manufacturing Defect?" token, awarded by Boshomi."The World Burns" token, awarded by SamB."Mountain of Wealth" token, awarded by Ciencia_Al_Poder."Love" token, awarded by He7d3r.
Assigned To
None
Authored By
bzimport, Jul 4 2005

Description

Author: dbenbenn

Description:
See [[User:Dbenbenn/sandbox#Gallery_pipe_trick]], for example. A
link in the caption of a gallery image ending in |]] is broken.


Version: unspecified
Severity: major

Details

Reference
bz2700

Related Objects

There are a very large number of changes, so older changes are hidden. Show Older Changes

lambdav wrote:

More than 6 years after this bug as been reported, the bug is still there !
How many time should we wait for correction ?
Priority must not be Low !

lambdav wrote:

Priority = High because blocking another High priority bug.

Nx.devnull wrote:

(In reply to comment #67)

(In reply to comment #65)

> An extension can register a callback and then do pre-save transformation on its
> content - this is the best solution in my opinion, since only the extension
> knows which parts of the content are supposed to be transformed, e.g. in the
> case of the gallery tag only the description part.

I am not too enthusiastic about encouraging extensions to do their own
arbitrary pre-save transform processing. In the longer run, general support for
extensions working with parsed (and pre-save transformed) wikitext / HTML DOM
input could be cleaner and should offer better opportunities for integration
with a visual editor. I am working on something like that in the Parsoid parser
(http://mediawiki.org/wiki/Parsoid). There is also a simple and undocumented
method for something vaguely like this in the PHP parser (the
setTransparentTagHook method).

However, I do not have a good solution for partly-parsed and -transformed
extensions like gallery. Will give this some thought, and ask others about
their ideas.

If the problem is that extensions could do arbitrary processing instead of the standard PST, then the only solution I can think of is to allow extension to somehow specify which part of their input should be transformed, and then let the parser do the processing. Implementing that would probably be much harder and more cumbersome.

The only thing that needs to be done on my patch is that I'm not sure the extension calling parser->preSaveTransform is 100% safe. I can give it a go if you want, but I don't want to waste my time if the patch will be rejected because my approach (letting the extension do the PST) is not the correct one.

New parsers are nice and all, but they do not fix the problem that we have right now, and they probably never will if you want to keep the current gallery syntax.

bluehairedlawyer wrote:

Actually David's wrong. This bug is seven not six years old. If its not fixed soon we'll have to figure out who's going to pay for it to go to college...

What would be wrong with running the all the contents of a <gallery> through preSaveTransform? I realise it's not wikitext but then preSaveTransform isn't the full parser? What particular pre-save transforms would corrupt the gallery syntax?

Nx.devnull wrote:

(In reply to comment #76)

Actually David's wrong. This bug is seven not six years old. If its not fixed
soon we'll have to figure out who's going to pay for it to go to college...

What would be wrong with running the all the contents of a <gallery> through
preSaveTransform? I realise it's not wikitext but then preSaveTransform isn't
the full parser? What particular pre-save transforms would corrupt the gallery
syntax?

I don't know off the top of my head what characters are allowed in filenames (in particular, is [, { or ~ allowed), but in theory:

<gallery>
File:Oh noez [[This is not a link.png|]] yeah, I know, this is a contrived example
</gallery>

There's also replacing tildes, substing and the reverse pipe trick. Replacing tildes would be the most realistic problem (if ~ is allowed in filenames)

[ { are both disallowed in any page name (which includes images). Three or more ~'s in a row is also forbidden in page titles. So you don't have to worry about that.

Personally I favour an approach of adding an interface for adding parserHooks that allows specifying options (such as if the parser hook should allow PST).

Actually David's wrong. This bug is seven not six years old. If its not fixed
soon we'll have to figure out who's going to pay for it to go to college...

That's still a good 11 years away. We're cross that bridge when we come to it.

bluehairedlawyer wrote:

Some parser tests

Attached:

bluehairedlawyer wrote:

Allow extensions to specify tag contents can be passed through preSaveTransform without being escaped

This is an alternative to Nikola's patch. It allows extensions and internal setHook functions to specify that there contents can be run through the preSaveTransform function without being escaped.

It passes all the current parser tests and a few others I've managed to throw at it.

Attached:

sumanah wrote:

bluehairedlawyer, could you put this patch in Gerrit?

bluehairedlawyer wrote:

Dan Collins has already done this (which is just as well as I haven't figured out how to do it yet). Here's the url:

https://gerrit.wikimedia.org/r/#/c/8784/

That was a smart way to solve it.

Commented some concerns on gerrit.

patch moved to gerrit, removing patch need review.

ori added a comment.Sep 16 2012, 8:53 PM

Dan Collins has two patches, not one:
https://gerrit.wikimedia.org/r/#/c/8784/ (previously mentioned)
and
https://gerrit.wikimedia.org/r/#/c/8785/

  • Bug 42078 has been marked as a duplicate of this bug. ***

Somebdy should really give those patches some love, as apparently the original committer has disappeared.

Gabriel: As you assigned this report to yourself a year ago, do you plean to rework the two patches?

Gabriel: As you assigned this report to yourself two years ago, do you plan to rework the two patches? Or are they "somehow" obsoleted in the meantime?

Change 8784 abandoned by Siebrand:
(Bug 2700) Pre-save transform inside extension hooks

Reason:
I'm abandoning this patch set as it's been open for a long time without any outlook at the open issues being resolved.

If the submitter or anyone else would like to work on it again, this patch set can be re-activated by clicking "Restore Change". Please only do this if you are actually going to work on it immediately.

https://gerrit.wikimedia.org/r/8784

  • Bug 72079 has been marked as a duplicate of this bug. ***
Nemo_bis set Security to None.
SamB awarded a token.Apr 4 2015, 5:10 AM
SamB added a subscriber: SamB.

Change 8785 had a related patch set uploaded (by Paladox):
(bug T4700) Pre-save transform inside extension hooks

https://gerrit.wikimedia.org/r/8785

Change 8786 had a related patch set uploaded (by MarkTraceur):
Pre-save transform inside extension hooks

https://gerrit.wikimedia.org/r/8786

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 23 2015, 2:25 PM

I was tooling around with this, but...

https://www.mediawiki.org/wiki/Manual:Tag_extensions#How_do_I_render_wikitext_in_my_extension.3F

This section says that tag extensions will never have PSTs applied to them. So maybe this bug is WONTFIX and we should abandon all patches for it?

This bug is not only/mainly about tag extensions: see pipe trick, subst, signatures.

AdamCuerden raised the priority of this task from High to Unbreak Now!.Sep 24 2015, 8:58 AM

this has surely gone on long enough, with maqsses of lost resources.

Aklapper lowered the priority of this task from Unbreak Now! to High.Sep 24 2015, 9:37 AM

This task has not suddenly become more urgent than before, hence reverting the recent Priority change. See https://www.mediawiki.org/wiki/Phabricator/Project_management#Setting_task_priorities for more information. Thanks.

Meno25 removed a subscriber: Meno25.Feb 22 2016, 7:18 PM

Change 272916 had a related patch set uploaded (by Cenarium):
Pre-save transform inside extension tags

https://gerrit.wikimedia.org/r/272916

Elitre added a subscriber: Elitre.Apr 18 2016, 4:35 PM
Restricted Application added a subscriber: Luke081515. · View Herald TranscriptJul 10 2016, 4:00 PM
Pols12 awarded a token.Aug 1 2016, 9:55 AM
Pols12 added a subscriber: Pols12.
He7d3r merged tasks: T10125: Pre-save transform not applied to markup inside <ref> container, T8222: any links inside <gallery> won't autopipe, T8157: template substitution does not work inside <ref> tags, T8814: Pre-save transform not executed for references (signatures, pipe trick, subst don't work), T10929: When a subst: is used inside a <ref> it doesn't get subst'ed, T12927: "pipe trick" links don't work, T14016: subst: don't work inside <ref> tags, T15072: refs fail to parse the piped link trick, T15959: Problem with <ref> and [[xxx (yyy)|]], T16007: Auto-piping does not work in galleries, T15120: New parser breaks signatures inside gallery tags, T16550: Cannot substitute templates withing <ref> tags, T17547: Automatic conversion of the wikitext doesn't work in references, T17467: Pipe trick not working when used as template parameter that is in turn used in a reference citation, T20064: The "pipe trick" does not work in footnotes, T22359: Subst: prefix does not work in image captions inside <gallery></gallery> tags., T15141: Pipetrick doesn't work in gallery image caption, T24258: Five tildes don't produce timestamp in en.wikipedia article namespace, T26131: ~~~~ in <gallery>, T26160: Magic words, templates and parser functions not substituted in <ref> tags, T26446: Automatic signatures (~~~~) do not work in <ref></ref>, T28859: Template substitution doesn't work inside <ref> tags, T30680: Pipe trick doesn't work within ref, T31775: <ref>...</ref> in template, T32135: Substitution not working in gallery tags, T36999: <ref> doesn't expand triple-brace template parameters, T37954: Inter-project links into <ref> do not work when alternative text is empty, T44078: {{subst:foo}} does not work in <ref>, T74079: {{subst: doesn't work inside <ref>...</ref>.Sep 3 2016, 12:53 PM
He7d3r added subscribers: bzimport, Wtmitchell, Krinkle and 14 others.
Izno added a project: Cite.Sep 3 2016, 7:25 PM
Izno moved this task from Unsorted backlog to Defect backlog on the Cite board.Sep 6 2016, 9:25 PM
demon removed a subscriber: demon.Oct 17 2016, 8:11 PM
Liuxinyu970226 awarded a token.
Wargo added a subscriber: Wargo.Apr 19 2017, 5:42 PM
Krinkle removed a subscriber: Krinkle.Apr 19 2017, 7:30 PM

Am I correct in thinking that {{subst:CURRENTMONTH}} and {{subst:CURRENTYEAR}} not being substituded in this edit is due to this bug?

Yes, substituting within refs won't work if this bug is not resolved!

@Thryduulf use {{#tag:ref|}} or {{subst:#tag:ref|}} and it will work

and I would love to see this added to the projects for fixing that are coming up. It is truly time that this got resolved for all wikis.

@Billinghurst add an entry to https://meta.wikimedia.org/wiki/2017_Community_Wishlist_Survey (I guess it would fit in the editing section?).

Liuxinyu970226 awarded a token.
Dalba added a subscriber: Dalba.Jan 26 2018, 1:27 PM
Daimona added a subscriber: Daimona.