Page MenuHomePhabricator

Langlinks endpoint is not returning articles for no.wikipedia.org
Closed, ResolvedPublicBUG REPORT

Description

How many times were you able to reproduce it?

Every time

Steps to reproduce

Go to https://en.wikipedia.org/w/api.php?action=query&format=json&llinlanguagecode=en&lllimit=500&llprop=langname%7Cautonym&prop=langlinks&redirects=&titles=Norway

Expected results

There's an item for the "no" language code, which represents the Norwegian article for Norway.

Actual results

There is not an item for the "no" language code. Note that there is one for Norwegian Nynorsk ("lang": "nn"), whose article is here.
This causes a missing Norwegian language option in the iOS app. I also see the same missing item when querying from nn.wikipedia.org via https://nn.wikipedia.org/w/api.php?action=query&format=json&llinlanguagecode=en&lllimit=500&llprop=langname%7Cautonym&prop=langlinks&redirects=&titles=Noreg.

Environments observed

App version: 6.7.3
OS versions: 13.7
Device model: iPhone 7
Device language: EN

Affected articles?

  • Norway, United States at least

Event Timeline

"nb" is in that list. I don't have any iOS devices, so I can't test, but isn't Norwegian Bokmål in the language option list?

@jhsoby nope, we just have Norwegian and Norwegian Nynorsk. I believe it's populated by https://meta.wikimedia.org/w/api.php?action=sitematrix&format=json&formatversion=2&origin=*, and it looks like we're ignoring specials.

Yes nb is there, and that is the correct language code for no.wikipedia.org. Norwegian is one of the cases where the domain name does not match the language code.

ApiQueryLangLinks.php even has a comment:

			// This is potentially risky and confusing (request `no`, but get `nb` in the result).
			$entry = [ 'lang' => $displayLanguageCode ];

@tstarling aah I see, I didn't realize we had cases like that. Thanks for clearing that up! I'll look into getting "nb" imported into our language list, that should fix it.

@tstarling actually before looking into adding the variance on our side, is this a case where https://meta.wikimedia.org/w/api.php?action=sitematrix&format=json&formatversion=2&origin=* should have it's "code" value changed to "nb" (at #205)? The site urls are there so we'd have both sets of info.

@jhsoby sorry for the delay - we should have everything we need to fix this on our end, thanks! We're pulling it into our current release.

First steps for iOS dev(s):

We need to replace the sitematrixURL in WikipediaLanguageCommandLineUtilityAPI.swift with https://meta.wikipedia.org/w/api.php?action=sitematrix&smsiteprop=url%7Cdbname%7Ccode%7Csitename%7Clang&formatversion=2&format=json" and use the lang property for populating languageName instead of code. lang has the nb distinction that we need. We'll also probably need to keep track of url from this response separately within Wikipedia.swift, and use that in MWKLanguageLink's siteURL computed property.

First steps for iOS dev(s):

We need to replace the sitematrixURL in WikipediaLanguageCommandLineUtilityAPI.swift with https://en.wikipedia.org/w/api.php?action=sitematrix&smsiteprop=url%7Cdbname%7Ccode%7Csitename%7Clang&formatversion=2&format=json" and use the lang property for populating languageName instead of code. lang has the nb distinction that we need. We'll also probably need to keep track of url from this response separately within Wikipedia.swift, and use that in MWKLanguageLink's siteURL computed property.

If it is of any help, the domain https://nb.wikipedia.org/ is a full redirect (don't know the proper term) to no.wikipedia. So any URL with nb instead of no will go to the correct place.

This fix is in TestFlight build 6.8.0 (1798).

@Tsevener I just tested the KaiOS app simulator, and it appears that this exact problem is happening there too. Would you mind notifying the team working on that about this task and how you fixed it?

@AMuigai we had a bug in the iOS app where our Article View > Choose alternate language > Language list filtered out the "Norwegian (Bokmål)" result even though it exists in no.wikipedia.org. It looks like this bug exists on KaiOS too.

This is because the subdomain code and language code differs for this wiki (subdomain is no.wikipedia.org and language code is nb). The results from the langlinks endpoint returns it under "nb" rather than "no", so if you are only looking for "no" you will have problems when filtering for an alternate language (see here in your codebase).

We fixed this by adding an additional property with "nb" to this object and adding the additional check when showing the article languages list. Depending on how your app is set up though your fix might be as clean as changing your code property here to nb, and whatever endpoints you are hitting will automatically redirect from nb.wikipedia.org to no.wikipedia.org without issue.

I wasn't able to file a bug to https://phabricator.wikimedia.org/project/profile/4305/ but feel free to reach out if you have any questions! Thanks!

Thanks @Tsevener and @jhsoby. I've filed it as a bug for us to look into.

JMinor claimed this task.