Mentioned in SAL (#wikimedia-operations) [2018-06-07T18:13:40Z] <subbu> Updated Parsoid (T183706, T192726, T194879, T196357, T196360, T43716)

No, round-tripping is the default. Specify source: null to explicitly disable it --- but the only reason to do so is to slim down the HTML a bit, maybe I don't even need to complicate the API for that.

The two requests example above is just for on-the-fly variant switching *while editing*. In that case you need to do a little dance instead of trying to convert directly from one variant to the other in order to ensure the round-trip information is preserved.

In most cases, you'd take the HTML from parsoid, stuff it into a JSON blob as original.html and then add updates.variant.target = 'my-target-variant' and send it to the pb2pb endpoint.

Or, you know, just ask for the variant you want directly from wt2html using the Accept-Language header...

I'm inclined to implement the pb2pb endpoint compliant with T114413 first, then if we find that there's a significant efficiency loss by JSON-encoding the HTML we can talk about adding a new specialized endpoint?

We've had a meeting with @cscott yesterday and here's a couple of notes from our discussion worths mentioning:

1. By default, we will return the "natural" variant - the HTML corresponding to the mixed-variant wikitext stored in the database. "By default" will be returned if no accept-language is provided, or if accept-language has a value that's not supported for a particular language.

2. Looking at the domain in RESTBase is not enough for splitting the Varnish cache or for deciding whether to even look at the accept-language and go to Parsoid for transformation, since LanguageConverter is actually enabled on all wikis, so, for example, even on English Wikipedia certain pages can have a different page language that will support conversion. This is mostly important for multi-language wikis like mediawiki.org. For cache-splitting Parsoid could provide the info about page language in some meta tag, however, for making a decision whether to go to Parsoid for transformation that's not very convenient, at least it's not easy to bootstrap, cause all pages must be re-rendered and re-stored in order for this to work reliably. I'm evaluating the possibility to include this info in the title_revision table so that RESTBase could decide on its own.

In my opinion, it would be the best to dump the whole LanguageConverter -{ }- markup, which is used to define specific variant translation for one term, and used the data from Wikidata instead. Wikidata can store language variant info and can be used across all wikimedia project, rather than volunteers maintaining the same CGroup across multiple project manually.

In T43716#5886163, @VulpesVulpes825 wrote:

In my opinion, it would be the best to dump the whole LanguageConverter -{ }- markup, which is used to define specific variant translation for one term, and used the data from Wikidata instead. Wikidata can store language variant info and can be used across all wikimedia project, rather than volunteers maintaining the same CGroup across multiple project manually.

See also the Glossary RFC (T484). Unfortunately glossaries tend to be topic-specific --- the dictionary you'd use for a pop culture article about movies may not be appropriate for a science article -- but the glossary could certainly source the variants from wikidata. It would be useful to be able to reference glossaries in a global manner as well, so that pages in zh in places other than zhwiki (ie, commons or mediawiki.org or wikimania.org etc) can use the constructed glossaries.

This ticket is important for the openZIM/Kiwix community and in particular its Chinese audience, see https://github.com/openzim/mwoffliner/issues/840