Page MenuHomePhabricator

Popups don‘t support language variant conversion and {{lang}} template
Closed, ResolvedPublic

Related Objects

StatusSubtypeAssignedTask
Resolved Pchelolo
Resolved mobrovac
ResolvedEevans
ResolvedEevans
ResolvedDzahn
ResolvedEevans
ResolvedEevans
OpenNone
DeclinedNone
ResolvedEevans
Resolvedfgiunchedi
ResolvedEevans
Resolved Pchelolo
OpenNone
OpenNone
OpenNone
Invalid GWicke
Resolvedliangent
Resolvedthiemowmde
OpenNone
Resolvedcscott
Resolvedcscott
Resolved Elitre
Resolvedcscott
Resolvedcscott
Resolvedcscott
Resolvedcscott
Resolvedcscott
OpenNone
DuplicateBUG REPORTNone
Resolvedcscott
OpenNone
OpenNone
OpenNone
OpenNone
ResolvedBUG REPORTJgiannelos
OpenNone
Resolved mobrovac
ResolvedJdforrester-WMF
ResolvedJdforrester-WMF
ResolvedEevans
ResolvedEevans
ResolvedEevans
Resolved mobrovac
Resolvedcscott
Resolved Pchelolo
Resolved Pchelolo
ResolvedABorbaWMF

Event Timeline

This is a general RESTBase limitation T159985, which is blocked on Parsoid at the moment. On the Parsoid side there has been some progress lately.

@Shizhao can you provide some replication instructions? Although I'm starting to learn Chinese, my Chinese is not good enough to understand what the problem is here :).

From what I understand the title "《POP TEAM EPIC》" was not showing in the preview (and 《》 instead, but I see the following when testing:

Screen Shot 2018-02-26 at 8.44.12 AM.png (258×351 px, 58 KB)
. Did a local edit happen to fix this?

Could you provide a test case maybe at https://en.wikipedia.beta.wmflabs.org/wiki/T188164 and https://en.wikipedia.beta.wmflabs.org/wiki/Link_to_T188164 ?

@bearND any theories?

@Shizhao can you provide some replication instructions? Although I'm starting to learn Chinese, my Chinese is not good enough to understand what the problem is here :).

From what I understand the title "《POP TEAM EPIC》" was not showing in the preview (and 《》 instead, but I see the following when testing:

Screen Shot 2018-02-26 at 8.44.12 AM.png (258×351 px, 58 KB)
. Did a local edit happen to fix this?

Yes, local edit via other template fixed it https://zh.wikipedia.org/wiki/Template:Nihongo_title

test case in zhwiki: https://zh.wikipedia.org/wiki/User:Shizhao/T188164

and see https://zh.wikipedia.beta.wmflabs.org/wiki/T188164 https://zh.wikipedia.beta.wmflabs.org/wiki/Link_to_T188164

@Shizhao and @Jdlrobson Unless if there's different configurations which I've not heard, contents in the first brackets are generally not shown (brackets are not shown too), e.g. by visiting Balloon loop and trying on tram or streetcar: "A tram is a rail vehicle which runs", where "(also tramcar; and in North America streetcar, trolley, trolley car, or cable car)" are ignored.

The problem regarding second brackets, where contents are also hidden but brackets shown, is interesting.

I've added the URL to the example page since it was not obvious to me from just looking at this task.

About the empty () in 《星色Girl Drop》():
The summary is derived from the Parsoid HTML. For this page this is https://zh.wikipedia.org/api/rest_v1/page/html/POP_TEAM_EPIC.
I was first thinking that the () was a remainder of removing some pronunciation but I now see that the Parsoid page has the same () in there.
In the wikitext I see a template for displaying Japanese words being used (Template:Nihongo: {{nihongo|《星色Girl Drop》|☆色ガールドロップ}}) . My current theory why the content between () is missing in the output is because Parsoid and RESTBase don't fully support language variants yet.

Mholloway added a subscriber: Mholloway.

I'm not entirely convinced the empty parens issue is a language variant issue. It might just be a Parsoid bug. From what I understand Parsoid leverages action=expandtemplates to get template-generated content, and it expands this wikitext just fine:

https://en.wikipedia.org/w/api.php?action=expandtemplates&text={{nihongo|%E3%80%8A%E6%98%9F%E8%89%B2Girl%20Drop%E3%80%8B%7C%E2%98%86%E8%89%B2%E3%82%AC%E3%83%BC%E3%83%AB%E3%83%89%E3%83%AD%E3%83%83%E3%83%97}}

Tagging Parsoid for investigation. If it's a separate issue as I suspect then it should probably get its own task.

The outer HTML for the "Girl Drop" section in https://zh.wikipedia.org/api/rest_v1/page/html/POP_TEAM_EPIC is

<span style="font-weight: normal;" about="#mwt31">(<span lang="ja"><span typeof="mw:LanguageVariant" data-mw-variant="{&quot;disabled&quot;:{&quot;t&quot;:&quot;☆色ガールドロップ&quot;}}"></span></span>)</span>

The blank section in the parens is the language variant markup. Your expandtemplates request is executing on en.wikipedia.org, instead of zh.wikipedia.org, so you're expanding the wrong template. This one contains the language converter markup:
https://zh.wikipedia.org/w/api.php?action=expandtemplates&text={{nihongo|%E3%80%8A%E6%98%9F%E8%89%B2Girl%20Drop%E3%80%8B%7C%E2%98%86%E8%89%B2%E3%82%AC%E3%83%BC%E3%83%AB%E3%83%89%E3%83%AD%E3%83%83%E3%83%97}}

Pchelolo added a subscriber: Pchelolo.

We're getting closer to launching language variants support in RESTBase, so I think we need to start the discussion about how page previews should be using the new API. Technically it's not hard, just add accept-language header to the summary request, but I guess it should be a product decision on which exact variant to request - the user's preferable, the one a current page is in now or other?

Per discussion on the Audiences-Services sync meeting, the initial step on the Page-Previews side could be to just explicitly set the Accept-Language header to the current wiki language code with no variant. This will prevent Varnish cache splitting for different browser/OS language settings and will unblock Services with deploying the summaries support of variants into production.

Afterward, as the product decision has been made on what exact variant to request for each situation, Reading web will be free to enable and test variants per-wiki in their own pace.

What's the plan for extracting the language code from the language inside endpoints? I see this middleware that looks interesting.
I'm using https://www.npmjs.com/package/accept-language

Note, we may have some troubles serving languages where the mediawiki codes do not match their standardise equivalents e.g. 'zh-classical', 'zh-min-nan',

See: https://gerrit.wikimedia.org/r/#/c/mediawiki/services/mobileapps/+/434193 for an example usage.

Per discussion on the Audiences-Services sync meeting, the initial step on the Page-Previews side could be to just explicitly set the Accept-Language header to the current wiki language code with no variant. <snip />

Broken out into T198619: Send the Accept-Language header with RESTBase API request for page preview requests.

What's the plan for extracting the language code from the language inside endpoints? I see this middleware that looks interesting.
I'm using https://www.npmjs.com/package/accept-language

Note, we may have some troubles serving languages where the mediawiki codes do not match their standardise equivalents e.g. 'zh-classical', 'zh-min-nan',

Yeah, I rejected the accept-language package for that reason in the work on https://gerrit.wikimedia.org/r/c/mediawiki/services/mobileapps/+/439997 but that's currently stalled.

@cscott's work on https://gerrit.wikimedia.org/r/c/mediawiki/core/+/443687 will allow MCS to pass either the industry standard or MediaWiki codes to the MW API and it "Just Works™". I'd expected to generalise whatever he writes into some JS for server-side code so that we can do the same with Parsoid, and then gently migrate all clients over to the industry standard and hide the MW weirdness as much as possible.

See: https://gerrit.wikimedia.org/r/#/c/mediawiki/services/mobileapps/+/434193 for an example usage.

Pchelolo claimed this task.

Everything is done here on the API side and on the popups side. For example, on sr.wikipedia.org the content of the preview respects user's variant settings.

However, Parsoid currently supports conversions only on a subset of wikis, so popups will get variants support as Parsoid adds more conversions.

Nothing really needs to be done here anymore, so I think we can close the ticket.