Page MenuHomePhabricator

RFC: HTML element for inline media from wikitext
Closed, ResolvedPublic

Description

  • Affected components: MediaWiki core, default wikitext parser.
  • Engineer(s) or team for initial implementation: Parsing Team.
  • Code steward: Parsing Team.

Motivation

RFC T118517 approved use of semantic HTML5 <figure> tags for media content, as described in Parsoid's spec.

However, <figure> elements are not considered "phrasing content" in HTML5, and therefore do not capture the proper semantics in all contexts in which media can be used in wikitext. More importantly, the HTML parsing spec causes paragraphs to break around <figure> (as would <div> or any other tag which is not "phrasing content" in HTML5), so we can not use it for inline figures without breaking rendering.

One solution to that would be for the parser to use <figure-inline> tags instead for media in contexts where phrasing content should be used.

The "-inline" suffix comes from the HTML4 block vs inline distinction, where "inline" roughly corresponds with HTML5 phrasing content. At this point, it can be thought of more as a wikitext construct.

Currently, the Parsoid service (as non-default MW parser) already implements this spec as well as a custom <figure-inline> element, and that seems to be working well in practice. However, while custom elements are permitted by the standard, there is a concern that we're picking an overly broad name that may eventually lead to conflict.

This RFC is about deciding whether and what element to use in the default parser of MediaWiki core (e.g. by it to both the legacy and Parsoid parsers, or by applying it to Parsoid before it becomes the default).

Requirements
  • Decide what HTML structure to use for inline media content.

Exploration

Some options to consider,

  1. Use a vendor prefixed name instead for phrasing content, like <mw-figure-inline>
  2. Use a vendor prefixed name in all contexts, instead of the standard <figure>, for all around consistency and to avoid breaking paragraphs
  3. Push to get <figure-inline> standardized first, since there might be more goodwill if go that route before shipping it
  4. Revert to using a non-semantic <span> tag for phrasing content
Proposal A: Option 3

The Parsing-Team--ARCHIVED would like to champion going forward with option 3 <figure-inline> tag as is. It has a shorter-term goal of unifying the legacy wikitext parser (MediaWiki-Parser) with Parsoid, and getting their media output to match is one step along that road. Bringing a proposal to a standards body is good idea but we should arrive with use cases, not necessarily solutions.

Proposal B: Option 4 (On Last Call)

[…]
The Parsing-Team--ARCHIVED has agreed to revert to using <span> tags for this purpose, with the condition that we switch to <figure>s in galleries and rewrite CSS rules to ensure we're using typeof attributes instead of tag names.

Thanks for helping think this through.

Event Timeline

Arlolra created this task.May 1 2020, 7:36 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 1 2020, 7:36 PM
ssastry updated the task description. (Show Details)
Arlolra updated the task description. (Show Details)May 1 2020, 8:59 PM
Arlolra updated the task description. (Show Details)May 2 2020, 2:00 AM
Arlolra renamed this task from [WIP][RFC] Use <figure-inline> for inline media to [RFC] Use <figure-inline> for inline media.May 4 2020, 8:25 PM
Arlolra added a project: TechCom-RFC.
Arlolra updated the task description. (Show Details)
Krinkle renamed this task from [RFC] Use <figure-inline> for inline media to RFC: Use <figure-inline> for inline media.May 5 2020, 11:11 PM
Krinkle updated the task description. (Show Details)
Krinkle added a subscriber: Krinkle.

I've updated the task description to use the new rubric for RFC tasks. Feel free to edit is as you see fit :)

Krinkle renamed this task from RFC: Use <figure-inline> for inline media to RFC: Decide on HTML element for inline media from wikitext.May 5 2020, 11:12 PM
Krinkle renamed this task from RFC: Decide on HTML element for inline media from wikitext to RFC: HTML element for inline media from wikitext.
Arlolra moved this task from P1: Define to P4: Tune on the TechCom-RFC board.May 6 2020, 6:20 PM
cscott updated the task description. (Show Details)May 11 2020, 4:53 PM

@Arlolra I noticed you skipped over phase 3. Did the outreach happen outside Phabricator? If so, it'd be good for transparency to document here who was invited/included etc.

Although it wasn't approved at the time, the proposal was discussed as part of RFC T118517.

Since v1.6.0, <figure-inline> has been part of Parsoid's spec. In order to roll that out, in T118520, all the Parsoid consuming clients needed to be made compatible.

My assumption was that, between the two, a group of both core and Parsoid developers had been exposed to the idea.

A broader media structure convergence page exists at https://www.mediawiki.org/wiki/Parsing/Media_structure

Milimetric moved this task from P4: Tune to P1: Define on the TechCom-RFC board.May 13 2020, 8:44 PM
Milimetric added a subscriber: Milimetric.

Took a look at this today, would love some more concrete requirements as is customary in phase 1 (https://www.mediawiki.org/wiki/Requests_for_comment#Phase_1:_Define). Once we have those, we can help facilitate exploration of the options. And we realize that requirements may change slightly based on the options we're exploring, that's fine.

cscott added a subscriber: cscott.EditedMay 13 2020, 9:28 PM

There is a lengthy discussion in T118517, which went through the (old) RFC process, and T118520: Use <figure-inline> instead of <span> for inline figures.. See T118517#5128632 and following. There is also a description in https://www.mediawiki.org/wiki/Specs/HTML/1.5.0#Images of the need to use some element other than <figure> for inline media (and then https://www.mediawiki.org/wiki/Specs/HTML/2.1.0#Images describes the solution adopted).

This is a funny RFC, since it was discussed and approved in part under the old RFC process, and the consensus from that first RFC process was implemented in Parsoid. This RFC is "blessing" the rest of the Parsoid format being migrated to the core/legacy parser, which would happen in any case "by magic" as the legacy parser is replaced. But it would aid our migration effort to be able to decouple this DOM structure change from the Parsoid roll out, so we'd like to get this change made in the legacy parser first and give the communities as much time as possible to adapt to the change before we start switching read views over to Parsoid (T55784: [EPIC] Use Parsoid HTML for all page views, https://phabricator.wikimedia.org/tag/parsoid-read-views/).

We have a patch for the consensus results of the first RFC (T118517, https://gerrit.wikimedia.org/r/507512) but it's felt that the communities would appreciate it if we made both the figure and inline figure changes at the same time, so they could lint for them together and weren't faced with two different figure-migration tasks in short order. So effectively merging the patch implementing the consensus outcome of the first RFC is currently blocked on resolving this one.

Some of the lack of consensus for the first RFC was due to browser-support issues, which the intervening 5 years have resolved. Let me quote from T118517#2710039:

Relationship of T118520: Use <figure-inline> instead of <span> for inline figures. was discussed. The rationale for T118520 seems to be that document.createElement('figure'); is required for IE6-8 compatibility of <figure> (this task) and so if you've got to add a custom element anyway, you might as well add the document.createElement('figure-inline') line at the same time so that you can use a more semantic element for inline figures.

Note that you can't use <figure> for inline figures because <figure> will break a <p> context:

> div = document.createElement('div');
> div.innerHTML = "<p><figure><figcaption>"
> div.innerHTML
"<p></p><figure><figcaption></figcaption></figure>"

We've been shipping the shim in our <head> to allow <figure> since 2017 (T122965):

<!--[if lt IE9]><script src=".../html5shiv.js"></script><![endif]-->

The code which allows <figure> is https://github.com/aFarkas/html5shiv/blob/51da98dabd3c537891b7fe6114633fb10de52473/src/html5shiv.js#L216-220. It's helpful to have an extra line of JS to tell html5shiv to register <figure-inline> (or whatever) as well, although T118520#1802441 claims it's not necessary. See https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/370206/9/includes/OutputPage.php .

Repeating here:

The direction of using <figure> for block images was approved in 2015 in an ArchCom IRC meeting which was prior to our current process around "Last Call" process. […] The IRC meeting notes from 2015 do specify an unresolved sub problem about inline images and the idea of <figure-inline> which does not yet have consensus. […]

@Krinkle: Could you explain the reasoning for placing this RFC in phase 1, given previous discussion on this topic (which you quoted from above)? Based on https://www.mediawiki.org/wiki/Requests_for_comment#Phases it seems to me that:

Phase 1: occurred during the earlier RFC process, when the requirements and possible solutions were discussed in depth (as per my recap), although consensus on the treatment of inline figures was not reached.

Phase 2: Affected components are Parsoid and the core parser. Software engineer(s) are the parsing team, with @Arlolra taking the lead. The code steward is again the parsing team, as per https://www.mediawiki.org/wiki/Developers/Maintainers.

Phase 3: Exploration and concrete proposal. The parsing team have has had various discussions related to <figure> over the past five years, and we have a concrete proposal, <figure-inline>, with patches in gerrit for the core parser and implementations already merged and in production for Parsoid, Visual Editor, Content Translation, and other downstream consumers for several years now.

Phase 4: Tune. This is where the parsing team believes we are now. "This is also a good time for stakeholders to confirm that any concerns they raised about the general direction have been accommodated. Review the proposal(s) against the Architecture Principles and identify any areas that warrant improvement or further discussion." We're opening this up for additional comments from stakeholders and TechComm.

As you were aware of the previous discussion and status and were present in the triage meeting, according to https://groups.google.com/a/wikimedia.org/forum/?hl=en#!topic/arch-forum/MK167bI9Lto, any further explanation of the status of this RFC, as TechCom sees it, would be helpful.

ssastry triaged this task as High priority.May 21 2020, 6:03 PM
Krinkle added a comment.EditedMay 27 2020, 8:47 PM

Notes from last week:

  • What's the expected benefit of a dedicated element? For example, does this enable particular optimisations or something within Parsoid's codebase? As a devil's advocate question, does that mean we should use dedicated elements for other wikitext concepts as well? E.g. <external-anchor>, <gallery-slideshow>, <transclusion>, etc. Assuming not, what makes inline images different?
  • If there is a long-term preference toward each concept having its own tagname, using a prefixed name (mw-) seems like a safer option and avoids conflicts. Not just with standards but also other libraries and frameworks we might want to adopt in the future. It's seems high risk to clobber that space by default. The industry has generally frowned upon browsers or frameworks requiring use of non-standard elements that are not namespaced/prefixed in some way.
  • The proposal describes <span> as "non-semantic". In what way is figure-inline considered semantic? To whom? In particular, does this mean rel="" attributes are not semantic? Parsoid uses those a lot to convey semantics between MW/Parsoid and its consumers. Does that not apply to inline images?
Base added a subscriber: Base.May 27 2020, 11:13 PM

Notes from last week:

  • What's the expected benefit of a dedicated element? For example, does this enable particular optimisations or something within Parsoid's codebase?

I think the idea was that, for the purpose of accessibility, all media on a page would be marked up as figures. However, since the figure element is in the wrong HTML5 content category for media that should be rendered inline, a different solution was needed for those cases. figure-inline was meant to convey the same meaning as the figure tag, just avoid breaking paragraphs.

But that doesn't seem right. Inline media don't seem to be being used as figures, for self-contained, illustrative purpose, where they occur onwiki. So that doesn't hold much water.

As a devil's advocate question, does that mean we should use dedicated elements for other wikitext concepts as well? E.g. <external-anchor>, <gallery-slideshow>, <transclusion>, etc. Assuming not, what makes inline images different?

A standard element already exists for what was supposed to be the intended purpose.

  • If there is a long-term preference toward each concept having its own tagname, using a prefixed name (mw-) seems like a safer option and avoids conflicts. Not just with standards but also other libraries and frameworks we might want to adopt in the future. It's seems high risk to clobber that space by default. The industry has generally frowned upon browsers or frameworks requiring use of non-standard elements that are not namespaced/prefixed in some way.

There is no such long-term preference, other than continuing to adopt tags that have been standardized.

  • The proposal describes <span> as "non-semantic". In what way is figure-inline considered semantic? To whom? In particular, does this mean rel="" attributes are not semantic? Parsoid uses those a lot to convey semantics between MW/Parsoid and its consumers. Does that not apply to inline images?

In the same sense that a <div> and <figure> differ.

Notes from last week:

  • What's the expected benefit of a dedicated element? For example, does this enable particular optimisations or something within Parsoid's codebase?

I think the idea was that, for the purpose of accessibility, all media on a page would be marked up as figures. […]

As a devil's advocate question, does that mean we should use dedicated elements for other wikitext concepts as well? E.g. <external-anchor>, <gallery-slideshow>, <transclusion>, etc. Assuming not, what makes inline images different?

A standard element already exists for what was supposed to be the intended purpose.

  • If there is a long-term preference toward each concept having its own tagname, using a prefixed name (mw-) seems like a safer option and avoids conflicts. Not just with standards but also other libraries and frameworks we might want to adopt in the future. It's seems high risk to clobber that space by default. The industry has generally frowned upon browsers or frameworks requiring use of non-standard elements that are not namespaced/prefixed in some way.

There is no such long-term preference, other than continuing to adopt tags that have been standardized.

These apply to <figure>, not <figure-inline>.

  • The proposal describes <span> as "non-semantic". In what way is figure-inline considered semantic? To whom? In particular, does this mean rel="" attributes are not semantic? Parsoid uses those a lot to convey semantics between MW/Parsoid and its consumers. Does that not apply to inline images?

In the same sense that a <div> and <figure> differ.

<figure> has semantic meaning from the HTML5 spec. <figure-inline> does not. It is a non-standard element with the same default semantics of an inline element as <span>, no different from <mw-xyz>.

If the only benefit for the custom element is that of standardisation, then I think we should shy away from it in our page view output until and unless such a standard has emerged. We can certainly push for that with our W3C membership and contacts at WHATWG, but unless we want to block this RFC on that, I think for this RFC the only point in favour of <figure-inline> is compat for existing Parsoid clients. Would you agree?

I vaguely recall that we've done this once or twice in the past through a content negotiation header to ease that transition with an inline conditional on the Parsoid side. I understand that's non-trivial, and there'd be no rush for Parsoid to switch (other than to happen sometime before adopting in page views).

But as long-term strategy in context of this RFC, the other three options are more compelling.

Arlolra closed this task as Declined.Jun 10 2020, 6:45 PM

<figure> has semantic meaning from the HTML5 spec. <figure-inline> does not. It is a non-standard element with the same default semantics of an inline element as <span>, no different from <mw-xyz>.

If the only benefit for the custom element is that of standardisation, then I think we should shy away from it in our page view output until and unless such a standard has emerged. We can certainly push for that with our W3C membership and contacts at WHATWG, but unless we want to block this RFC on that, I think for this RFC the only point in favour of <figure-inline> is compat for existing Parsoid clients. Would you agree?

I agree that the custom element currently has no meaning that would enhance accessibility and that it would need to be standardized or widely adopted to do so.

But, given what I said above in T251641#6189168, I don't even think inline media are being used as figures, so I wouldn't bother advocating for it with those bodies.

I vaguely recall that we've done this once or twice in the past through a content negotiation header to ease that transition with an inline conditional on the Parsoid side. I understand that's non-trivial, and there'd be no rush for Parsoid to switch (other than to happen sometime before adopting in page views).

But as long-term strategy in context of this RFC, the other three options are more compelling.

The Parsing-Team--ARCHIVED has agreed to revert to using <span> tags for this purpose, with the condition that we switch to <figure>s in galleries and rewrite CSS rules to ensure we're using typeof attributes instead of tag names.

Thanks for helping think this through. I'm going to close the RFC since we're no longer doing anything controversial.

I think as a process question it would be better if TechCom accepted and resolved this issue, blessing the solution described above (<span> tags in articles and <figure> in galleries) and also taking ownership of starting a discussion at the W3C level. This will certainly not be the last time mediawiki makes changes to its output, and it would be good to at least settle the question: would the W3C prefer we deploy experimental features first, then bring that experience to a discussion at the W3C; or vice-versa? That will impact our process in the future, and define the relationship of TechCom with the W3C. It also provides an opportunity for TechCom to establish its role as a "facilitator" instead of a "roadblock", which isn't so much an issue for our team but seems to be a factor in why some projects apparently avoid bringing their architecture decisions through the RFC process. I don't really like how it looks that the resolution of our figure-inline discussion was to decline the RFC and so bypass TechCom; I'd rather resolve this affirmatively.

Agreed @cscott. I suggest re-opening this and moving it to 4, narrowing in on "Option 4: Use <span>."

If there are benefits we missed I would have expected Parsing Team to have brought those up by now. And given it is what we do today in MediaWiki I'm not sure there is much space for blocking concerns about it, as they are what we do today.

If you re-open I think we can put this on last call the week after.

Arlolra reopened this task as Open.Jun 10 2020, 8:51 PM
Arlolra moved this task from P1: Define to P4: Tune on the TechCom-RFC board.
ssastry assigned this task to Arlolra.Jun 11 2020, 6:03 PM
ssastry lowered the priority of this task from High to Medium.Jun 25 2020, 6:02 PM

@Krinkle Please bring this to last call whenever possible.

Krinkle updated the task description. (Show Details)Jul 8 2020, 8:15 PM
Krinkle moved this task from P4: Tune to P5: Last Call on the TechCom-RFC board.

I've updated the task description to reflect the latest consensus. This RFC is now on Last Call for two weeks. Unless new concerns are raised, option 4 will be accepted on Wed 22 July 2020.

Change 623834 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/core@master] Inline media will use spans

https://gerrit.wikimedia.org/r/623834

Krinkle removed a subscriber: Krinkle.Sep 2 2020, 6:02 PM

Change 623906 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] s/figure-inline/span/

https://gerrit.wikimedia.org/r/623906

Change 623834 merged by jenkins-bot:
[mediawiki/core@master] Remove figure-inline from the set of allowed tags in the Sanitizer

https://gerrit.wikimedia.org/r/623834

Change 641800 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Shore up serializing spans for inline media

https://gerrit.wikimedia.org/r/641800

Change 641800 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Shore up serializing spans for inline media

https://gerrit.wikimedia.org/r/641800

Change 644393 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a18

https://gerrit.wikimedia.org/r/644393

Change 644221 had a related patch set uploaded (by C. Scott Ananian; owner: Subramanya Sastry):
[mediawiki/vendor@wmf/1.36.0-wmf.20] Bump wikimedia/parsoid to 0.13.0-a18

https://gerrit.wikimedia.org/r/644221

Change 644393 merged by jenkins-bot:
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a18

https://gerrit.wikimedia.org/r/644393

Change 644221 merged by jenkins-bot:
[mediawiki/vendor@wmf/1.36.0-wmf.20] Bump wikimedia/parsoid to 0.13.0-a18

https://gerrit.wikimedia.org/r/644221

Change 623906 merged by jenkins-bot:
[mediawiki/services/parsoid@master] s/figure-inline/span/

https://gerrit.wikimedia.org/r/623906

Arlolra closed this task as Resolved.Dec 16 2020, 9:06 PM

Parsoid started producing spans again in version 2.2.0 of its output.

Client adoption is tracked in T266143

Coming up with a new spec for galleries that includes the switch to figures is in T268250. For now, it's using spans in place of the previous figure-inlines since that's looking like a larger effort.

The RFC was approved so I'm going to consider this resolved. The open questions about TechCom / W3C in T251641#6212609 live to fight another day.

Change 651570 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a21

https://gerrit.wikimedia.org/r/651570

Change 651570 merged by jenkins-bot:
[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.13.0-a21

https://gerrit.wikimedia.org/r/651570