Implement <gallery> extension natively inside Parsoid
Closed, ResolvedPublic

Description

As described in T110909, extensions that process wikitext need native Parsoid equivalents so that the HTML markup for wikitext from those extensions is similar to the rest of the page, and also so that references that are page-global are handled properly.

This requires T55874 to be resolved as well.

Related Objects

ssastry created this task.Aug 31 2015, 5:12 PM
ssastry updated the task description. (Show Details)
ssastry raised the priority of this task from to Normal.
ssastry added a project: Parsoid.
ssastry added a subscriber: ssastry.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 31 2015, 5:12 PM

https://github.com/Wikia/mediawiki-services-parsoid/pull/5/files is the wikia gallery extension implementation which we might be able to start with.

ssastry moved this task from Backlog to Next Up on the Parsoid board.Dec 17 2015, 5:44 PM
Arlolra claimed this task.Jan 13 2016, 11:59 PM
Arlolra moved this task from Next Up to In Progress on the Parsoid board.

Change 264026 had a related patch set uploaded (by Arlolra):
WIP: Native <gallery> extension

https://gerrit.wikimedia.org/r/264026

Change 264026 had a related patch set uploaded (by Arlolra):
T110910: Native <gallery> extension

https://gerrit.wikimedia.org/r/264026

Arlolra added a subscriber: cscott.Jan 21 2016, 2:44 AM

@cscott Going to be working on the spec here: P2506

The spec contains both a wikitext (in data-mw) and HTML representation of the gallery. How does Parsoid know which one to use when converting back?

Testing it in only appears to take note of the data-mw. Isn't the purpose here that VE can edit the generated HTML directly?

@Esanders Thanks for taking a look. I was too hasty in removing the WIP prefix.

You're right that currently serialization in the patch only considers the extsrc. Once we agree on an HTML representation that'll go away. Your feedback there would be appreciated (bearing in mind the TODOs in the head of the paste).

In the DOM spec invalid lines are dropped, however if we get rid of data-mw then we need to preserve that information somewhere. Maybe as in-place HTML comments?

Good point. An HTML comment seems reasonable (and editable!). I've updated the spec and patch accordingly.

That said @cscott might have an opinion on using HTML comments - it may add some unnecessary complexity around escaping certain characters.

Change 264026 had a related patch set uploaded (by Arlolra):
WIP: Native <gallery> extension

https://gerrit.wikimedia.org/r/264026

I think we should avoid comment nodes because of T95040.

Also making it editable should be given carefully consideration given that you are essentially allow the user to write wikitext. If these are simply labelled as comments, what if the user includes '</gallery>' in one of them?

@Esanders Good points.

In the case of,

<references>
stuff that isn't a ref
</references>

we drop the "stuff that isn't a ref".

See "References: 5. ref tags in references should be processed while ignoring all other content" in parserTests.txt

We should probably be consistent with that.

Arlolra moved this task from In Progress to Next Up on the Parsoid board.Apr 8 2016, 5:27 PM
Arlolra moved this task from Next Up to In Progress on the Parsoid board.Jul 13 2016, 4:49 AM
Arlolra raised the priority of this task from Normal to High.

Is the spec at P2506 still what we're aiming for? (At the moment I'm working on T150621 to work with the current patch set.)

P2506 may have gone stale. Let me clean that up for you.

Is the spec at P2506 still what we're aiming for?

Yup, updated. Please let me know if you have any question.

Is the spec at P2506 still what we're aiming for?

Yup, updated. Please let me know if you have any question.

Separately, @Arlolra and I were also discussing yesterday about what kinds of finer-grained support VE might need / benefit from. For example, would data-mw attributes at the level of individual images (vs. at the level of a gallery) be useful? I imagine you will figure this out better based on experience with the current output.

For the purpose of finer-grained control of dirty diffs on edits to galleries, we might add data-parsoid attributes and element ids on individual images. T151367 is related to that.

Change 264026 merged by jenkins-bot:
Native <gallery> extension

https://gerrit.wikimedia.org/r/264026

Arlolra closed this task as Resolved.Nov 22 2016, 9:56 PM

Mentioned in SAL (#wikimedia-operations) [2016-12-07T21:36:38Z] <arlolra> updated Parsoid to version 3cf19c6b (T110910, T102209, T94949, T150112, T151570, T149209, T150213)