Right now Flow sends body_only requests to Parsoid and doesn't store full HTML documents, only the <body> tag. But this means we're throwing away information like the Parsoid version number, as well as the base URL.
We should store full documents for new content, while remaining backwards compatible with old content that's body-only. Then in the future we can write a maintenance script that re-parses old content to the new format.
Change 486303 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/Flow@master] WIP: Save full HTML documents, load only body in VE mode
Change 486304 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/Flow@master] WIP: Store full HTML documents for Flow content
I've created two WIP patches for this which use different approaches, see above. In short, version A (486304) adds a new $body_only parameter to Utils::convert() and friends, which defaults to false. The Flow Parsoid Utils API has a new body_only parameter added, and it's set to true in the context of rendering a Flow post in VE. If you don't do this, the wikitext conversion to HTML results in meta, head, etc being in the textarea.
version B (486303) is much simpler: by default and without exception we remove the body_only param from the request to Parsoid. in mw.flow.ve.Target.prototype.loadContent, we specify that we want section number 0 when parsing the document. This approach is much simpler (2 lines changed) but results in a lot more HTML getting sent over the wire (see below). I'm also not totally sure if we need to also load other section numbers, although from a quick review it seems like that would be OK.
But this means we're throwing away information like the Parsoid version number, as well as the base URL.
Are these the only other two pieces of data we need from the full HTML document? As it stands, storing the full HTML document will result in something like the following getting stored in rev_content:
<!DOCTYPE html> <html prefix="dc: http://purl.org/dc/terms/ mw: http://mediawiki.org/rdf/"><head prefix="mwr: http://localhost:8080/index.php/Special:Redirect/"><meta charset="utf-8"/><meta property="mw:pageNamespace" content="2600"/><meta property="mw:html:version" content="1.8.0"/><link rel="dc:isVersionOf" href="http://localhost:8080/index.php/Topic%3AUswaduwev13mg7qt"/><title></title><base href="http://localhost:8080/index.php/"/><link rel="stylesheet" href="//localhost:8080/load.php?modules=mediawiki.legacy.commonPrint%2Cshared%7Cmediawiki.skinning.content.parsoid%7Cmediawiki.skinning.interface%7Cskins.vector.styles%7Csite.styles%7Cext.cite.style%7Cext.cite.styles%7Cmediawiki.page.gallery.styles&only=styles&skin=vector"/><!--[if lt IE 9]><script src="//localhost:8080/load.php?modules=html5shiv&only=scripts&skin=vector&sync=1"></script><script>html5.addElements('figure-inline');</script><![endif]--><meta http-equiv="content-language" content="en"/><meta http-equiv="vary" content="Accept"/></head><body data-parsoid='{"dsr":[0,35,0,0]}' lang="en" class="mw-content-ltr sitedir-ltr ltr mw-body-content parsoid-body mediawiki mw-parser-output" dir="ltr"><section data-mw-section-id="0" data-parsoid="{}"><p data-parsoid='{"dsr":[0,35,0,0]}'><span about="#mwt1" typeof="mw:Transclusion" data-parsoid='{"pi":[[]],"dsr":[0,12,null,null]}' data-mw='{"parts":[{"template":{"target":{"wt":"SITENAME","function":"sitename"},"params":{},"i":0}}]}'>local</span> <a rel="mw:WikiLink" href="./Main_Page" title="Main Page" data-parsoid='{"stx":"simple","a":{"href":"./Main_Page"},"sa":{"href":"Main_Page"},"dsr":[13,26,2,2]}'>Main_Page</a> <i data-parsoid='{"dsr":[27,35,2,2]}'>blah</i></p></section></body></html>
Should we store a subset of the full HTML document that contains only the essential pieces of information?
We should store full documents for new content, while remaining backwards compatible with old content that's body-only.
Could you elaborate on what exactly is required here for backwards compatibility? AFAICT, both patches I submitted will handle displaying and editing body-only content, as well as new full HTML document content. However, I might be missing other scenarios.
Change 486303 abandoned by Kosta Harlan:
WIP: Save full HTML documents, load only body in VE mode
Yes, good idea. Maybe we can store only the <body> tag. Right now we store the contents of the body tag, but if we instead store the entire body tag (including attributes), we get most of the important information, and we can encode any other information we need as data attributes on the body tag.
We should store full documents for new content, while remaining backwards compatible with old content that's body-only.
Could you elaborate on what exactly is required here for backwards compatibility? AFAICT, both patches I submitted will handle displaying and editing body-only content, as well as new full HTML document content. However, I might be missing other scenarios.
If there is old content stored in the DB that is body-only, it might be served up without ever flowing through a code path that converts or body-wraps it, and so it'd be sent to the client as body-only, which I think we want to avoid.
Change 486304 merged by jenkins-bot:
[mediawiki/extensions/Flow@master] Store Parsoid version and base URL in Flow body content