In T284389#7224349, @Ladsgroup wrote:

In T284389#7224233, @Esc3300 wrote:

Shouldn't this remain open until the import is actually done?

No. That's not the job of syadmins to import the data.

I see that UniversalLanguageSelector is loaded, but for some reason I don't see the language menu at the top (like on wikisource.org where it says "English" if that's the current language, and you can click it to change languages and change ULS settings). Are ULS webfonts are on by default ("download fonts when needed")? That will be important.

I'm not sure if that would work because that would require parsing the text in a new format that we would have to define -- for example, newlines might be significant in this format whereas they weren't previously. There are probably also multiple possible ways to resolve the file name -- could be on another wiki, for example. I think it makes more sense to just use the existing wikitext facility for making links. Someone could make a tool to generate the wikitext from a list like you gave, though.

Thanks for explaining the issues with PDF. I expected it would be something like that but it's good to see it all spelled out.

In T278623#6956024, @Languageseeker wrote:

For example, https://en.wikisource.org/wiki/Index:Lippincotts_Monthly_Magazine_51 has 1042 pages that you would need to number individually instead of being to set the numbering using page ranges.

In T278623#6952226, @Languageseeker wrote:

Update, while reading the documentation for Proofread page (1). I discovered that you can transclude an image sequence using the format <pages index="Index Name" from="Start Image" to="End Image"/> even if the images are in a different format. This implies that at some point, the code does numerically sequence the image files. However, on the Index ns <pagelist /> does not work. This implies that the Index ns is not aware or make usage of the numerical sequence that exists.

In T47779#6798724, @Ilham.nurwansah wrote:

Thank you @Joseagush for adding me into this thread. The Balinese-Latin converter indeed so usefull! Thanks also to @kamholz for bringing the source-code here. I will adapt it for Latin-Sundanese transliteration.

In T47779#6790286, @Joseagush wrote:

Hi @kamholz, one of Sundanese Wikipedia community ask me to help this ticket for Javanese script (and Sundanese script as well). I asked your help as your same experience when adding Balinese script which already deployed last year. Let me know if you need help from the wikipedia community. At least, you can give some advice here what must they prepared for Transliteration/LanguageConverter. Thanks in advance.

Just created a new task for this.

In T263082#6732938, @Fomafix wrote:

Better is to the use always the BCP 47 conform language codes:

<span lang="sr-Cyrl"><langconvert from="sr-Latn" to="sr-Cyrl">zdravo</langconvert></span>

Joseagush explains the use case in the link to the talk page in the task description. The contributors in Bali are cataloging a large number of manuscripts and converting to PDF is an additional step which is not trivial for them and has resulted in decreased image resolution with their attempts so far. I'm not going to say there is no way to get around this but I also think it's important to make Wikisource accessible to smaller communities like this. They've made a lot of progress already in learning the technical aspects of Wikimedia sites. Just keep in mind that their capacity is still limited, so if they need to learn any new processes (such as img2pdf which would require them to use Python on Windows, use the command line, etc.) then we should be sure it's really needed.

In T263082#6697435, @Johan wrote:

Added to https://meta.wikimedia.org/wiki/Tech/News/2020/52 – please let me know if there are any mistakes in the text.

I did a quick test and it seems to work. Haven't tested with a proper OAuth library.

Is there a way to cross-link Wikisource pages across editions, similar to how Wikipedia links articles? Does that even make sense?

Unfortunately I don't have a lot of time to work on it at the moment. There are at least two things that would be involved in porting it:

Any chance we can get this merged soon?

I just tested this locally. The issue arises from ProofreadPage's code in getImageWidth. If the width is not set in the index, the default (self::DEFAULT_IMAGE_WIDTH) is 1024. This is presumably meant to avoid excessively large image files, but it's counterproductive in this case.

Looking into this a bit further, I'm starting to agree with @Xover that this is not a very good way of achieving what I want. Among the issues:

Here is an example of how it looks on Palmleaf.org currently. It will not look exactly like this on Wikisource of course. Sections like Leaf 1, Leaf 2, Leaf 3 will correspond to pages in the Page namespace. The content prior to the "auto-transliteration" heading is what editors will type, and the transliteration will be added below. It should be interleaved like this, page by page, so that readers don't get lost. Given that, it makes sense to me to make it part of the parser output for each page. This makes transclusion work without further effort, and it means that editors can preview the output while editing the page (which, at least in the case of the Balinese work, definitely helps their proofreading efforts).

In T260211#6386654, @Xover wrote:

@kamholz I don't understand what it is you're proposing to do here, nor see how it will have applicability outside just Balinese content. From whence comes #transliterate and what does it do? Why hard-code <br> inside ProofreadPage and provide two copies of the text? Why can this not be done with a normal template?

This is related to something PanLex is currently doing in a gadget I've recently ported from Palmleaf.org. There's a community in Bali that's been doing Balinese palm-leaf manuscript transcription there and it's in the process of being moved to Wikisource. The manuscript scans all come from IA. I've already batch uploaded them to Commons using PDFs from IA.

I've just uploaded a new patchset that splits out ban-latn. There are now three codes:

Thanks! Yes, I can see how this would be a useful feature to have. It's not currently needed for the Balinese work I am doing, but I can imagine it could be needed in the future for the annotation use case. Annotation/correction of manuscripts is certainly a thing people do.

In T245359#6365255, @Nemo_bis wrote:

Option 1 means we don't do any Bali script. To have both scripts, one of which default, we do as said in option 2. A recent minimal example is b39b3944dc8.

The languages need to be configured properly and we'll also need to move the messages to the locale with their real script.

In T259645#6364873, @Soda wrote:

I don't think the language field should be completely unvalidated. In its current state, a user can (hypothetically) enter bn, Bengali, bengali, bangla, Bangla ( all of which refer to the Bengali language ) but only one of which will result in the desired change of the page language (bn).

I think an OOUI select widget which presents the user with a list of available languages while also allowing arbitrary user input could be a possible way to go. That would work in preventing users from making mistakes such as the above while also allowing users to specify unknown/unsupported languages if they need to.

I've now had a chance to discuss this with Joseagush, one of the main coordinators Balinese Wikipedia. His strong preference is for option 1: ban should be Latin-only and ban-bali should be for Balinese script. His argument is that most Balinese language online is in Latin script, and most Balinese people expect Latin and may not be comfortable with Balinese script. Another advantage is that this maintains compatibility with the coding used on ban.wikipedia.org, so that work on Balinese Wikisource (which will contain a good amount of Balinese script) will not unnecessarily interfere with existing Balinese content on Wikimedia. (Incidentally, this also seems to be how Javanese works: jv is Latin script and jv-java is Javanese script.)

New patchset is uploaded now. There is currently no validation of the Index page's language code, but that is arguably the preferable behavior, because Wikisource editors can add sources in whatever language they want for whatever reason they want, and thousands of languages will not be in the list known to LanguageNameUtils::getLanguageNames. To allow this freedom I think the Index field should not be further validated. It isn't currently possible to set the page language to a language not in the Names.php list (I guess for good reason? don't know), but at least it can be recorded accurately in the Index this way.

Based on @Tpt's comments on Gerrit, it looks like it's a lot cleaner to override getPageLanguage in the PageContentHandler to return the language based on the current Index value, and not modify page_lang in the database. Working on an updated patchset now -- there are a couple other things to address beyond that.

It's still possible to override the pagelang on individual pages in the Page namespace by using Special:PageLanguage as usual. Also, if you change the Index's langcode and the pagelang of anything in a corresponding Page doesn't match, it's left alone. Example:

I think I see a fix for this, but you'll have to test it on toolforge -- shall I submit another PR on GitHub?

Must be an issue with views/template.twig, where it does this:

I haven't designed a MediaWiki-specific input method for Balinese script. I'm not sure what should be done for T245360: Add input method for ban-Bali (Balinese in Bali script). I did design a Keyman keyboard. The Keyman stuff is highly context-sensitive and the rules are pretty complex. What's the format/language for input methods?

FYI the latest proposal has been submitted as a project grant: https://meta.wikimedia.org/wiki/Grants:Project/PanLex/Balinese_palm-leaf_transcription_platform_on_Wikisource

Based on recent comments in the above-referenced GitHub issue for oauth2-server, it looks like current best practice is not to allow callback prefixes of any kind. That means it's just the OAuth consumer proposal form that needs to be updated.

The easily fixable issue I had was that I had registered the callback URI as http://localhost:3000 but the library I was using was passing it to the server as http://localhost:3000/, which was enough to cause a mismatch. I registered a new consumer with the slash on the callback URI and it works now.

I've resolved some of the issues and turned remaining issues into separate tasks.

OK, I took several steps just now to investigate this and found the following:

I just tried making a new OAuth 2.0 consumer called archiveleaf which is not owner-only. It's not approved yet (in fact I requested it not be approved since it's being tested) but my understanding is it should still allow me to authenticate as myself (user Lautgesetz). I get the same error.

By the way, there's the possibility of using the ArchiveLeaf extension to work with palm-leaf manuscript collections in other languages. There are no solid plans for that currently but it's something PanLex is investigating. Just wanted to mention that it may be useful beyond this project.