Primarily a Parsoid problem.

Looking at this briefly,

! url

at https://github.com/wikimedia/parsoid/blob/master/lib/wt2html/pegTokenizer.pegjs#L1262 is probably wrong. It's not a valid reason to push preproc broken, the preprocessor doesn't know about that.

The fix looks similar to https://github.com/wikimedia/parsoid/commit/0890c9ba61f0e108231bd6d54e759c7f3d9cf303, however everything is complicated by needing to reparse for extlinks.

A hacky workaround of trying to pop when you get an extlink followed by a "]" at,
https://github.com/wikimedia/parsoid/blob/master/lib/wt2html/pegTokenizer.pegjs#L1258
is going to get messy because of suppressing pipes in different context (extlink in wikilink context).

Some simple examples,

{{1x|[[http://hi.com |ho]]}}

vs

{{1x|[http://hi.com |ho]}}

@cscott Wanna sanity check this analysis?

Change 365064 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] [WIP] T170289: Accept url in wikilink target position while tokenizing

https://gerrit.wikimedia.org/r/365064

In T170289#3426679, @Arlolra wrote:

@cscott Wanna sanity check this analysis?

In the patch @Arlolra sez:

The argument I'm making is that the preprocessor would see these as wikilinks, so it's not correct to bail on encountering a url. What's more, a pipe needs to be treated as in wikilink context (see the test), which complicates trying to work around this in the tokenizer.

I think this is fundamentally correct.
https://github.com/wikimedia/mediawiki/blob/master/includes/parser/Preprocessor.php#L46 only recognizes [[, not [, and as far as I can tell there's nothing in https://github.com/wikimedia/mediawiki/blob/master/includes/parser/Preprocessor_DOM.php which treats [[ ... ] specially. From the preprocessor's perspective, this is a broken wikilink and it emits [[ (but see T172306 for a wrinkle).

I think the approach that handles this via the

broken_wikilink
  = &"[[" &{ return stops.push('preproc', 'broken'); }
    a:("[" (extlink / "[")) { return a; }

clause is the right one: the broken token should definitely remain on the preprocessor stack. OTOH, [[http://example.com]] is *not* a broken link; we'd need to parse that as a wikilink and convert it to an extlink and surrounding brackets at a later stage.

Change 365064 merged by jenkins-bot:
[mediawiki/services/parsoid@master] T170289: Accept url in wikilink target position while tokenizing

https://gerrit.wikimedia.org/r/365064

Mentioned in SAL (#wikimedia-operations) [2017-08-03T18:03:38Z] <arlolra> Updated Parsoid to 651f12c2 (T119802, T73386, T170289)