Page MenuHomePhabricator
Create Task
Maniphest T363279

/page/summary/ and /page/mobile-html/ should handle BCP47 languge code for wikidata description
Open, Needs TriagePublicBUG REPORT
Actions

Description

The app is sending BCP47 language code to the endpoints:
For Chinese (Taiwan) is:
zh-Hant-TW, but the Wikidata only accepts the old one: zh-tw.

Steps to replicate the issue (include links if applicable):

What happens?:
The Wikidata endpoint does not support BCP47 language code, and will fallback to zh-hant.

Accept-Language: zh-Hant, zh-Hant-TW
臺灣電視劇

Accept-Language: zh-TW
台灣電視劇

What should have happened instead?:
Should show the zh-tw one when requesting the endpoint with Accept-Language: zh-Hant-TW.

Accept-Language: zh-Hant
臺灣電視劇

Accept-Language: zh-TW, zh-Hant-TW
台灣電視劇

Related Objects
Search...

Event Timeline

cooltey created this task.Tue, Apr 23, 10:53 PM
cooltey mentioned this in T343387: /page/summary/ and /page/mobile-html/ do not get the latest article description in zhwiki.
cooltey added a comment.Tue, Apr 23, 11:09 PM

Notice an issue, not sure if it is related to the cache.

When running it locally, it works correctly
http://localhost:8888/zh.wikipedia.org/v1/page/mobile-html/八尺門的辯護人

1713913685031.jpg (390×1 px, 128 KB)

But it shows the incorrect one on production
https://zh.wikipedia.org/api/rest_v1/page/mobile-html/八尺門的辯護人

1713913645604.jpg (399×1 px, 161 KB)

Func updated the task description. (Show Details)Wed, Apr 24, 4:31 AM
Func updated the task description. (Show Details)Wed, Apr 24, 4:53 AM
Shizhao moved this task from Backlog to Apps/Tools/Services on the Chinese-Sites board.Sun, Apr 28, 3:58 AM
Jgiannelos added a comment.Mon, Apr 29, 12:34 PM
  • When asking page/mobile-html internally directly to mobileapps node service, accept-language works as expected
    • curl -v "https://mobileapps.svc.codfw.wmnet:4102/zh.wikipedia.org/v1/page/mobile-html/%E5%85%AB%E5%B0%BA%E9%96%80%E7%9A%84%E8%BE%AF%E8%AD%B7%E4%B
    • curl -v "https://mobileapps.svc.codfw.wmnet:4102/zh.wikipedia.org/v1/page/mobile-html/%E5%85%AB%E5%B0%BA%E9%96%80%E7%9A%84%E8%BE%AF%E8%AD%B7%E4%B -H Accept-Language: zh-Hant-TW
  • The same requests to restbase level fail:
    • curl -v restbase1031.eqiad.wmnet/zh.wikipedia.org/v1/page/mobile-html/%E5%85%AB%E5%B0%BA%E9%96%80%E7%9A%84%E8%BE%AF%E8%AD%B7%E4%BA%BA
    • curl -v restbase1031.eqiad.wmnet/zh.wikipedia.org/v1/page/mobile-html/%E5%85%AB%E5%B0%BA%E9%96%80%E7%9A%84%E8%BE%AF%E8%AD%B7%E4%BA%BA -H Accept-Language: zh-Hant-TW

The latter also fails with cache-control: no-cache that forces pregeneration of mobile-html content. I think restbase doesn't pass the headers properly to the backend

Jgiannelos added a comment.Mon, Apr 29, 1:25 PM

After some debugging it looks like restbase overrides the submitted accept-language with this:
https://github.com/wikimedia/restbase/blob/master/lib/language_variants_filter.js#L15

Which eventually means that accept-language is always zh regardless if we send zh-Hant-TW

cooltey merged a task: T344675: Chinese language conversion is wrong in Wikipedia App.Fri, May 3, 8:40 PM
cooltey added a subscriber: SunAfterRain.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL