Page MenuHomePhabricator
Create Task
Maniphest T343387

/page/summary/ and /page/mobile-html/ do not get the latest article description in zhwiki
Open, MediumPublicBUG REPORT
Actions

Description

Steps to replicate the issue (include links if applicable):

What happens?:
Both page/summary and page/mobile-html are not showing the latest description.

What should have happened instead?:
Should show the latest description "2023年台灣公視電視劇系列" in Wikidata.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Make language codes lowercase to catch description from Wikidata description correctlymediawiki/services/mobileappsmaster+7 -5
Customize query in gerrit

Related Objects
Search...

Event Timeline

cooltey created this task.Aug 2 2023, 9:42 PM
Restricted Application added subscribers: Stang, Aklapper. · View Herald TranscriptAug 2 2023, 9:42 PM
Shizhao added a project: Chinese-Sites.Aug 3 2023, 5:58 AM
MSantos triaged this task as Medium priority.Aug 3 2023, 2:06 PM
MSantos edited projects, added Page Content Service; removed Content-Transform-Team.
cscott subscribed.Aug 3 2023, 2:08 PM

I wonder if this would work if you used Accept-Language: zh-hant-tw instead of zh-tw? Both ought to be valid, but the former is the "more correct" BCP-47 code.

cooltey added a subscriber: Dbrant.Aug 4 2023, 12:07 AM

Hi @cscott,

It does not work if I apply Accept-Language: zh-hant-tw when requesting the APIs.

@Dbrant helped to purge the page (八尺門的辯護人) today and the issue is still there, we can confirm that it is not a cache issue.

Shizhao moved this task from Backlog to Apps/Tools/Libs/Services on the Chinese-Sites board.Aug 4 2023, 2:50 AM
MSantos added a project: Essential-Work.Aug 21 2023, 2:37 PM
cooltey added a parent task: T356615: [Epic] Return the correct Chinese language variant in the REST endpoints.Feb 5 2024, 5:56 AM
cooltey added a subscriber: Jgiannelos.Apr 6 2024, 12:21 AM

Found the root cause of the issue:

Related to this ticket: T327379

The description in wikidata is still using the old language code, for example, the Chinese (Taiwan) is zh-tw, but the new mapping json has zh-TW, which is uppercase on the TW.
https://github.com/wikimedia/mediawiki-services-mobileapps/commit/73ada17fe18e0d3c770b9eaf1b7c4f75b03f3fbe

This will cause an error when trying to make a request for the code in the mwapi.getMetadataForSummary, and it will return multiple descriptions.

https://www.wikidata.org/w/api.php?format=json&formatversion=2&errorformat=html&errorsuselocal=1&action=wbgetentities&props=descriptions&titles=%E5%85%AB%E5%B0%BA%E9%96%80%E7%9A%84%E8%BE%AF%E8%AD%B7%E4%BA%BA&sites=zhwiki&languages=zh-TW

vs

https://www.wikidata.org/w/api.php?format=json&formatversion=2&errorformat=html&errorsuselocal=1&action=wbgetentities&props=descriptions&titles=%E5%85%AB%E5%B0%BA%E9%96%80%E7%9A%84%E8%BE%AF%E8%AD%B7%E4%BA%BA&sites=zhwiki&languages=zh-tw

cc @Jgiannelos

(I am still trying to fix it but if it looks like an easy fix for you, please help to advise, thanks!)

gerritbot added a comment.Apr 9 2024, 3:42 AM

Change #1017960 had a related patch set uploaded (by Cooltey; author: Cooltey):

[mediawiki/services/mobileapps@master] Make language codes lowercase to catch description from Wikidata description correctly

https://gerrit.wikimedia.org/r/1017960

gerritbot added a project: Patch-For-Review.Apr 9 2024, 3:42 AM
Jgiannelos moved this task from Backlog to To Deploy on the Content-Transform-Team-WIP board.Apr 10 2024, 6:07 PM
gerritbot added a comment.Apr 10 2024, 6:09 PM

Change #1017960 merged by jenkins-bot:

[mediawiki/services/mobileapps@master] Make language codes lowercase to catch description from Wikidata description correctly

https://gerrit.wikimedia.org/r/1017960

cooltey claimed this task.Apr 10 2024, 8:50 PM
Jgiannelos moved this task from To Deploy to To Verify on the Content-Transform-Team-WIP board.Apr 23 2024, 1:36 PM
cooltey added a comment.Apr 23 2024, 9:34 PM

The app is sending BCP47 language code to the endpoints:

For Chinese (Taiwan) is:
zh-Hant-TW, but the wikidata only accepts the old one: zh-tw.

Steps to reproduce:
add Accept-Language: zh-tw or Accept-Language: zh-Hant-TW when requesting https://zh.wikipedia.org/api/rest_v1/page/mobile-html/不夠善良的我們

to see the difference.

cooltey added a comment.Apr 23 2024, 10:54 PM

A related ticket is created: T363279

Maintenance_bot removed a project: Patch-For-Review.Apr 26 2024, 7:03 PM
Jgiannelos removed a project: Content-Transform-Team-WIP.Apr 11 2025, 1:37 PM

This is going to be revisited after PCS is completely migrated away from RB

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits