A problem with existing machine translation for Japanese like Google/Bing/DeepL is that they would halucinate and omit context from original text when performing translation. And this from my personal experience occur quite a bit more frequent than even other langauges that machine translation are deemed to have similarly worse performance like Chinese. And it is also much more often for those machine translation engines to try top do summary instead of actually performing translation when doing Japanese compared to other languages. My personal experience is that these NMT based translation engine are even more likely to cause these problems than requesting translation from Generative AI, and I am not saying Generative AI are performing great either.

Is it an intended effect of this ticket or is it a bug that now when I access https://maps.wikimedia.org/ from browser by entering the URL into browser URL bar directly, I get an error saying

Request from 221.127.107.156 via cp5026 cp5026, Varnish XID 729746413
Upstream caches: cp5026 int
Error: 403, Forbidden: Map tiles are restricted to Wikimedia & affiliated sites only. See https://wikitech.wikimedia.org/wiki/Maps/External_usage if you believe your usage supports the Movement. at Sat, 04 Mar 2023 12:24:42 GMT

?

In T162331#8121512, @MPhamWMF wrote:

Closing out low/est priority tasks over 6 months old with no activity within last 6 months in order to clean out the backlog of tickets we will not be addressing in the near term. Please feel free to reopen if you think a ticket is important, but bare in mind that given current priorities and resourcing, it is unlikely for the Search team to pick up these tasks for the indefinite future. We hope that the requested changes have either been addressed by or made irrelevant by work the team has done or is doing -- e.g. upgrading Elasticsearch to a newer version will solve various ES-related problems -- or will be subsumed by future work in a more generalized way.

In T236593#8092471, @LucasWerkmeister wrote:

It’s still not clear to me which problem the -x-Q123-1 patch is trying to solve. Several languages have been mentioned in this task, but which of them would benefit from this system? I feel like for several of them, we’ve already reached the conclusion that separate forms are in fact the way to go.

I’d like to extract a general rule from @Fnielsen’s comment above (T236593#5610903): if you need separate statements, then you need separate forms or lexemes. (I think this is a sufficient condition, though it might not be a necessary one.) Pronunciation (whether pronunciation audio or IPA transcription) is probably the most significant kind of statement here: if a speaker would pronounce the spellings differently, then they should be different forms – regardless of whether the difference is a completely different ending as in octopuses/octopi, or just an extra schwa as in aft(e)nen. (I don’t find the hyphenation example as convincing… don’t you need a different hyphenation for every spelling variant, even for cases that really should just be multiple representations of one form? E.g. co‧lor/co‧lour – that could just be multiple statements on the same form, with different monolingual text language codes.)

I suspect this rule covers the Norwegian example that originally motivated this task: I feel like “parametere” and “parametre” are probably pronounced differently, much like “aftnen” and “aftenen” are pronounced differently in Danish according to Finn. For Vietnamese chữ Nôm, I feel like @mxn’s comment at T236593#8024999 goes in a similar direction, though I admit I find the whole Chinese-characters part of this discussion hard to follow.

For the cases where you really only want to have one form with multiple representations, I still agree with @daniel’s comment (T236593#5610378): “you make up a code for each of the spellings”. In practice, the only way to “make up a code” that we currently support is to append -x-Q12345 to an existing, established language code. As far as I understand, this solution works well for Hebrew: e.g. ספר/סֵפֶר (L67105) (the “book” word) uses the language codes he and he-x-Q21283070, where Q21283070 represents Tiberian vocalization, the orthography with diacritics. At some point, an editorial decision was made that the spelling without diacritics “deserves” the unsuffixed he language code (instead of both spellings using an -x-Q12345 language code), which I think is reasonable: data reusers who don’t care about the different spellings can use the most standard language code (he) and its single representation per form.

Allowing people to append an integer number to the item ID adds a second way to make up a code, and one that seems less useful to me: without knowing what the number means, how do I know which form representation to use? To me this runs counter to the goal of “allow[ing] the consumer to choose which variant they prefer”. For the languages that appear to need multiple representations for the same language code per form (e.g. the Indian languages @Mahir256 mentioned in T236593#5608530?), is it not possible to make the item ID approach work, by creating more special-purpose items? Wikidata editors would then make a decision which of the possible spellings “deserves” the standard language code, and which additional items need to be created (“spelling with character X”, “spelling with sequence Y”?). I understand that not all languages have standardized spellings where you can use a single item ID to refer to the spelling variants of a wide range of lexemes (like in Hebrew), but I think it should still be possible to describe different spellings using items that carry more meaning than just a number.

In T236593#5610378, @daniel wrote:

I recall that we had long discussions about this when initially deciding on the data model. In technical terms, the question was whether we would allow only a single literal value for a spelling variant, or a list or set of words. Allowing a list or set would enable the kind of flexibility @jhsoby is asking for. But the down side is that it introduces ambiguity when listing forms (you would always have to list all of them, in undefined order), and when generating text (which one should you use)?

If I recall correctly, we decided that we want to give the consumer of the data maximum control over which variant they prefer, by forcing the producer to provide different variant codes for all different spellings. We had discussions about how to encode this in the variant (language) codes, and how to represent it in the UI, but decided to leave that for later.

So, the solution that we envisioned when originally discussing this about four years ago was: you make up a code for each of the spellings, in a way that allows the consumer to choose which variant they prefer. If that is done by encoding a region or a rhyme or a tradition or school or whatever will depend on the language. If it's a stylistic choice, name the style.

The same approach can be used for historical spellings. codes could look something like de-x-hist-nd-15jh or something (this code is totally made up and probably linguistically nonsense).

In T180345#7916540, @GerardM wrote:

Hoi,
We can only support scripts supported in Unicode.

In T180345#7916094, @mxn wrote:

Ken Lunde (2009) writes that chữ Nôm had a dedicated script code of Cu in ISO 15924:2004 (which has since been superseded by ISO 15924:2022). Does anyone have more information about these two-letter codes or why chữ Nôm didn’t get a four-letter code corresponding to this two-letter code?

"Hani" simply mean "Chinese[Han] characters".
"vi-Hani" mean "Vietnamese, written in Chinese[Han] characters".
Chu Nho, despite widely used in Vietnam in ancient time, are written according to Classical Chinese grammar, and as such should classify as Classical Chinese text, with code "lzh", similar to comparable works from Japan, Korea, and other neighboring regions.
The existence of ISO code Jpan is for the mixed use of Kana together of Kanji in Japanese text, which is still the common writing system for Japanese system nowadays.
The existence of the ISO code Kore is for the mixed use of Hangul together with Hanja in Korean text. Although Hanja's role in Korean language have greatly diminished, it is still not unexpected to see Hanja in modern Korean text, hence the code "Kore" which represent Hanja+Hangul is still the default code for Korean language writing system, at least in South Korea.
On the other hand, I do not think the mixed use of Han characters with other writing systems, say Latin alphabets, is an expected usage in Vietnam nowadays, hence I don't think it is necessary to apply for a new ISO 15924 code for such mixed use to reflect this.

In T305604#7837572, @Aklapper wrote:

@C933103: Hi, how is this different from T124168?

I have removed the Korean part of the ticket and focus on Vietnamese writing, due to problem of ambiguity of "Kore" script tag in ISO 15924, as mentioned in December 2020.

I *might* have encountered the same issue in Meta, where I do hnot have the GoogleTrans widget, but this cannot be reproduced reliably

I tested locally and if the site is read-only and you try to save your edit (assuming you opened the edit page before read-only), The response still contains the edit and you can simply click on the Submit button, nothing gets lost according to my test.

I found that, the cause of the event is likely due to https://en.wikipedia.org/wiki/User:Endo999/GoogleTrans this widget being enabled.

In T302147#7735454, @ppelberg wrote:

hi @C933103 – we appreciate you filing this task. To ensure the issue you are experiencing is NOT being caused by a gadget or script, can you please use repeat the steps you described in the task description in "safe mode"?

Instructions for enabling "safe mode" can be found here: https://www.mediawiki.org/wiki/Help:Locating_broken_scripts?useskin=vector-2022#Test_if_you_have_problems_related_to_user_scripts_or_gadgets.

Please let us know if you have any questions doing the above.

In T302170#7724288, @Aklapper wrote:

@C933103: I see. Which one, specific feature request is this task asking for?

In T302170#7724280, @Aklapper wrote:

For warmer colors and to make eyes more comfortable when using a screen (whether reading Wikipedia or not), many operating system have a "Night Light" setting to change the color temperature of the screen. I don't see why this is something you'd want to implement only for some website, server-side?

In T302148#7723533, @Dbrant wrote:

When reading an article, you can access the Talk page by tapping the overflow menu (button with the dots) in the toolbar, and selecting "Talk page".

But not the other way round. I have modified the task description accordingly.

So with the example from July 7 last year, why isn't this request proceeding?

In T26070#7670546, @Aklapper wrote:

This ticket is about dark-mode. No further ticket is needed.
For the time being, anyone is welcome to set browser.in-content.dark-mode to true in Firefox, or set Settings > Appearance > Theme to Dark in Chromium, if they wanted to reduce electricity, or monitor burn, or such.

In T301627#7706343, @Aklapper wrote:

I assume this is about ロシア (please always provide links / text that allows to reproduce). Cannot reproduce in Android 10, so I do not think that this is anything that the Wikipedia app itself can influence, but it is up to the fonts installed on your system instead:

Conversion table used in the wiki currently:
traditional to simplified: https://zh-yue.wikipedia.org/wiki/MediaWiki:Gadget-tradsim-trad2simcore.js
simplified to traditional: https://zh-yue.wikipedia.org/wiki/User:William915/Gadget-tradsim-src2tradcore.js

In T301625#7708412, @cooltey wrote:

Hi @C933103

Thanks for reporting this. I check the app and looks like the text under each language variant is correct. Could you please update your app version and check if it shows correctly?

Or, did you mention something different? If so, please let us know, thanks!

I think this part of the original ticket

Note that the same wishlist also have another user requesting a <del>dark</del> yellow mode. (Edit: Typo)

Note that Cantonese Wikipedia also experience similar phenomenon vs Chinese Wikipedia, according to on-site discussion record.

Note that in addition to multilingual wiki, there are also wikiprojects like Wikipedia for Min Nan, where same article on same subject are being written in same language in multiple different article using multiple script, due to technical difficulty making automatic conversion between these different articles impossible.
Currently, wikidata can only handle one site link to one script version of article on such sort of Wikipedia, and linking to another article through another wikidata QID entity, making article in the other script cannot be easily accessed through interlanguage link.

In T36217#6386871, @Liviojavi wrote:

Hi,
I am an Emilian speaker and eml Wiki user. the eml code is obsolete according to ISO, but I find it better than using split egl and rgn codes, or even using just egl (very offensive and disrespectful to Romagnol people). Linguists today find Emilian and Romagnol separate languages, when there are other tongues objectively further to each other, but considered dialects of the same language, like Maghreb Arabic and Yemenite Arabic: at times mutually unintelligible but paradoxally considered part of the same language. Emilian and Romagnol have a high rate of mutual intelligibility, and they are almost never mutually unintelligible if not for literally a bunch of words.

Me and the other eml Wiki editors use a particular page layout in which every single page can be eventually translated in each Emilian or Romagnol dialect, without "sacrificing" order and graphical harmony of the page. This pagination method is very helpful to us and for our languages survival: take for example the Neapolitan wiki (nap) and the Tarantine wiki (roa-tara), Tarantine is de facto a dialect of Neapolitan language, but nobody consults that Wiki, instead of the Neapolitan one, which is very lively (if we count how many Neapolitan speakers are there and how much are they interested to read and benefit of their own Wiki). If nap Wiki adopted our layout style and moved Tarantine translations into their pages, maybe the Tarantine pages would be way more read and edited, a deal for Tarantine survival.

In Italy we speak a lot of endangered local languages, Wikipedia is one of our saviors from language extinction. If you split Emilian and Romagnol, the latter risks to endanger itself even more, as rgn "subWiki" is one of the least translated in the eml "superWiki". We, Emilians, and Romagnols too, find very useful a united Wiki, as we can easily help Romagnols in some pages as Emilians and viceversa because, I repeat it, our two languages are VERY VERY mutually intelligible.

Thus, I ask you to not split our two languages just to adapt to the ISO, what benefits do you get from it? Our two languages, so insignificant for you non-speakers, but that mean a lot for us, will be harder to manage in their own new Wikis, as certainly you would split editors and simple users too. Nobody would notice the split, and according to us, eml Wikipedians, it's a useless and dangerous thing to do for our languages as I said before.

The "community that wants to split and rename the Wiki" you were talking about is formed by non Emilian-Romagnols (Dragonòt, Nickanc, Vituzzu, etc...), so whose opinion is worthier?
Then the last discussions about it (excluding this one) are from 2012: 8 years later eml Wiki is still united, but the split threat still impends.

Thanks for reading,

Liviojavi.

In T215032#7193390, @Esc3300 wrote:

If you use "mul", you wouldn't actually need "cmg".

In T180771#7193232, @Esc3300 wrote:

The request isn't complete: it lacks samples.

In T185858#7174704, @Verdy_p wrote:

THat's a limitation of Mapnik using bitmaps; we need other (dynamic) projections for poles and this usually requires vector format. Bitmap tiles using cyclindric projections (oriented around polar axis) are unusable above about 80 degrees, and have severe distortions about 60°.

First, according to my understanding, Wikimedia Maps doesn't use Mapnik
There were projects that used Mapnik to render arctic area by changing projection and modifying how features are rendered according to special characteristics of polar area. I am not sure whether the project still survive now, but it indicate it is noy that hard to achieve.

There are other renderers using vectorized tiles which allows easier navigation.
Still, editing maps around poles is difficult with iD or even with JOSM (without using custom plugins), and many people can't use something else. Adn there are still issues for representing objects crossing the international dateline (not solved at the OSM Serverside API, requires adaptation by the client editor to make necessary adaptations).

I think, the lack of renderer have inhibited editor software willingness to improve support for polar area rendering, and wikimedia maps as an openly available tile supporting it could help boost editors support
And nowadays editing users can edit information in the area using tools like Level 0.

Autonym for nan-hani:

Other than font size, font type should also be cutomizable, for people with different needs

and this must be easy to rename to a standard domain (languagecode.wikipedia.org) once the Language committee grants a full approval. And again, it's OK if it's just languagecode.wikipedia.org right from the start as long as patroling for vandalism works reasonably well.

Not really. There are multiple over-a-decade-old request on changing language code of a wiki and only one get achieved, and that one achieved renaming caused so much problems that all other requests are still not doable after they have been requested for over a decade
The T172035 have to be solved first

Should the task be spilt to two?

join the two tables together.

What I mean in term of "multiple language converter" is a script level conversion separate from vocabulary level conversion. Currently the language converter in Chinese Wikipedia actually merged two different things together to handle it, that's difference in script and different in vocabulary. It might be possible for a resident in China Maimland desiring to use Traditional Chinese but use Mainland China vocabulary, but the current language conversion scheme cannot cater such list and it wouldn't make sense to add a separate conversion table just for those minority users who might have complicated family and language leaening background and thus prefer different scropts from the one they currently use

Is it currently possible to offer conversion through language converter to convert between Indian Lakh and English Million?

I have just realized something.
Are those codes supposed to be case-sensitive?
Because, if so, then that pull request wouldn't work
And that's also what I am seeing

Why "consider" instead of just requesting it this way?

It seems like the custom dark vector css skin I am using now will become unusable in a while due to some breaking changes being made to css properties in mediawiki.
I think it's now a good time to introduce one of those existing skin update it and turn them ibto an official option.
(I am not really sure about why some people are arguing that it's going to be difficult when there are already something like that made by various users floating around for years)

As mentioned in the upstream issue, the Leaflet have a "rotate" branch that can do the required thing.

In T180771#5627406, @Davidzdh wrote:

在T180771#4544573中，@C933103写道：

And then for hak... Can someone verify that "Hakka (Traditional Han script)" and "Hakka (Simplified Han Script)" are proper way to describe how Hakka speakers would write their language in Han scripts?

Of course it is one of the correct way to write this language. Ministry of Education, ROC awards Literary Award of Taiwanese and Hakka (教育部閩客語文學獎, their website is https://www.edu.tw) every year. You can see hak-hant here.

Also, somewhat related, when rendering the map at https://maps.wikimedia.org/?lang=zh#15/10.6986/106.7430 , the name of openstreetmap relation 7157197 was rendered without an appropriate character, and resulted in a "tofu" empty box character instead of the character it's supposed to be rendered. The text are 茹𦨭县 which should be rendered as https://i.imgur.com/g3gfUew.png but it is currently rendered as https://i.imgur.com/S3AlPC2.png

There were definitely >100 languages affected if language fallback didn't work in all languages, but if it's only for those that fallback to zh, and a few other languages that utilize script tag in Mediawiki project but utilize an alternative language tagging style in OSM projects, then the scope of the task could have been smaller than anticipated.

See Also https://github.com/gravitystorm/openstreetmap-carto/issues/2208
And this also affect some other languages, see the link for further details

Is the system case-sensitive? Because the document use "zh-hans" and "zh-hant" while relevant keys in OSM are usually "zh-Hans" and "zh-Hant"

Example link: https://maps.wikimedia.org/?lang=zh#12/24.4542/118.0906
Current text shown: https://i.imgur.com/dc5O9J2.png
Expected text shown: https://i.imgur.com/CqJmSlN.png

Please consider the following email response given to @Liuxinyu970226 when they asked certain linguistic expert about their opinion on the matter: https://imgur.com/a/YT8bnzJ (I am not sure whether sufficient permissions have been obtained by the user for me to link the mail on the public internet but let's just look at it for now)

Well, as mentioned, the code cmg previously suggested as possible alternative is actually not appropriate according to email exchanges you have conducted with professors that know more about these terminology. And given the email exchange also confirmed that the current ISO language codes for Mongolian languages doesn't really make much sense either, it would also be wrong to use individual language code for such purpose. So following the convention already used by others should be the most sensible way to represent such text string in the wiki. But then, if certain member of Langcom stand firm on their position and unwilling to change, then no amount of sensibility can force them to change.

In T137810#4920450, @Liuxinyu970226 wrote:

@C933103 Because he said me "shame".

In T137810#4920436, @Liuxinyu970226 wrote:

@Badaa

ISO-639 is a total mess. What a shame!

https://www.mediawiki.org/wiki/Code_of_Conduct#Unacceptable_behavior states: 1. Personal attacks, violence, threats of violence, or deliberate intimidation.

Yes that could be a browser specific conversion as my Chrome browser is converting "ß" into "ss". Then again it shows that it is necessary to look at browser implementation on normalization instead of just standards.

Some characters, like "ß", "。", "｡" from the list linked above , would not normalize to become regular ASCII character even if the NFKC normalization is applied despite they can still be identified and converted into ASCII characters by browser. And the list was not extensive thus more characters could escape the NFKC normalization. Especially note worthy thing is that the two ideographic full stop would be treat as a dot by browsers and thus it can still be used to bypass blacklist of almost all url in spam blacklist.
And then for the longer term solutions, browsers may not follow standardized way in RFCs fully and might have their own way to normalize links for users and that might need to investigate behavior of each browsers

@Liuxinyu970226 Not sure why are you asking me this here when you yourself have stated that this seems to be an inappropriate place. Anyway I think it depends on projects and communities of each projects might also have different opinions

There are already some websites that sue Chinese-based captcha around, however it is a really really bad idea to me personally. As someone who can read and write Chinese and is also a native language user of one of the Chinese languages, it's very complicated for me to type Chinese text into computer dependent on environment and the input method available. Worse case scenario I would have to use Google Translate or Google Search to find out those matching characters over the internet and then copy them over in order to finish a localized captcha challenges. Please DO NOT implement such troublesome thing.

In T137810#4641351, @Popolon wrote:

Just a little tip, on firefox, go to about:config, then search for the browser.urlbar.decodeURLsOnCopy option and change it from false to true, this is essential for non pure-english characters (aka any accent or other char from any other language using latin script or any other script, so the url) :

https://zh.wikipedia.org/wiki/%E8%92%99%E5%8F%A4%E8%AA%9E%E6%96%B9%E8%A8%80%E5%88%97%E8%A1%A8

Thas is unreadable (will be the same with mongolian name in both cyrillic/mongolian script) will become

https://zh.wikipedia.org/wiki/蒙古語方言列表

shorter and readable (at least if you know the script).

@Popolon I believe Monguor and all that do not/no longer use Mongolian Script in writing so that's not really relevant to the context.

@Popolon According to my understanding assuming they are correct understanding, using Arabic as analogy, what you propose would be like making different monolingual value for "Libyan Modern Standard Arabic", "Egyptian Modern Standard Arabic", "Tunisian Modern Standard Arabic". Yes, Libyan/Egyptian/Tunisian Arabic are all different and could be considered as different languages, however there are only one single literary standard here. Surely, there are different phonetic literary standard that more closely reflect individual languages, like the Cyrillic alphabet being used to spell different Mongolic languages, which would warrant the establishment of wiki in each of their individual languages, however there are only one Classical Mongolian Script just like there are only one Modern Standard Arabic. You can say mvf is closest to Classical Mongolian in the same way as Egyptian Arabic being closest to the standard of Modern Standard Arabic, however they are not equal.

Sorry for late reply,
@Liuxinyu970226 If the concern of ISO639's RA is "users of the codes understand that part 2 of the standard has a code that includes several coded languages in part 3.", then probably what can be done is ask for cancellation of the mvf code and khk code in the ISO639-3?

I think I am able to edit lead section on mobile mediawiki site without js enabled using the js-less editor, because I have just tried that on English Wikipedia and it seems like it is working?

It's also an annoyance to me as well as my current user name is supposed to start with a small letter

https://zh.wikipedia.org/w/index.php?title=Wikipedia:互助客栈/技术&diff=51348491&oldid=51348410
All these characters can also be typed into Wikipedia as part of a url, bypassing blacklist, and then get accepted by browsers which would convert them into basic ascii alphanumeric characters, and then send users to the blacklisted webpage.

Actually my original ticket could be a little clearer...
Like clarifying that the "example" there was meant to mean there are articles in cdo/nan/hak wikipedia that are written in alternative script and thus there should be related monolingual code that would allow recording of those article names in wikidata language field.
Thus I would like to bump the request for monolingual language code cdo-hani and nan-hani.
And then for hak... Can someone verify that "Hakka (Traditional Han script)" and "Hakka (Simplified Han Script)" are proper way to describe how Hakka speakers would write their language in Han scripts?

1. If a new site is to be created for each incubator site, how will WMF turn them into a full site once they become eligible? Last time I heard about it, such redesignation seems to be very cumbersome and that's also why wp/yue and wp/nan still haven't be moved to the desired domain name after almost a decade from their initial proposal. Will it also take a decade for any new projects to get a full site if the proposal is to be adopted?
2. Is it going to lengthen the entire wiki creation process, and also requires more bureaucratic processes, as well as requiting more manpower to handle each and every applications? Now it is incubator→Full site, in the proposal it will be incubator→Experimental site→Full site.
3. Are those goals unachievable by overhauling incubator itself? It seems like Wikia is now going to change the url of their non-English wiki in order to save the SSL certification cost by changing urls in format of zh.community.wikia.com to community.wikia.com/zh, and each of these different language edition sites are still independent. Is that not achievable in Incubator?
4. Likewise, is it possible to create such new experimental site in a way as easy as creating a new wiki site on wikia?

The wikidata property proposal https://www.wikidata.org/wiki/Wikidata:Property_proposal/coordinate_location_GCJ02 would depend on this property datatype.