Page MenuHomePhabricator

Javanese OCR installation steps for Wikisource
Open, Needs TriagePublic

Description

Dear all,

WMID and Universitas Kristen Duta Wacana in Yogyakarta, Indonesia, are in partnership to develop OCR system for Javanese characters. This will help people to transcribe text, which in Javanese characters, easily in Wikisource.

They have already developed a closed software system, to read the characters automatically and produce the results which can be copy and paste. The software is still apart from MediaWiki software. That means, people will have to upload the pages to the software, wait to be processed, and copy the OCR result to Wikisource. Until now, that is the workflow I know.

They want to open their code to Wikimedia projects under GNU GPL v2, however, we do not know how to port the code into the Wikisource platform.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 19 2018, 11:32 AM
Rachmat04 updated the task description. (Show Details)Jul 19 2018, 11:34 AM
Amire80 added a subscriber: Amire80.

Hi @Rachmat04,

Thanks for creating the task.

I'm trying to add some relevant tags. Hopefully it gives it the right direction.

In theory, if the software is GPL, we can probably have our own instance of it, and create a service that reads it, but I'm really not an expert in how OCR and services work.

Pchelolo moved this task from Backlog to watching on the Services board.Jul 19 2018, 12:56 PM
Pchelolo edited projects, added Services (watching); removed Services.

The software is still apart from MediaWiki software. That means, people will have to upload the pages to the software, wait to be processed, and copy the OCR result to Wikisource. Until now, that is the workflow I know.

Community-Tech has worked on an OCR gadget for Wikisource for Indic languages in the past, to use Google's Vision API (as far as I understand that).

Please see T142770#2639235 and T120788 for more information. I'd hope that an implementation could be shared.

Niharika added a subscriber: Niharika.

Unfortunately Google does not support Javanese for its OCR service (also see T140037#2521503). We ran into this issue with an Indic language as well.

If and when Google supports it, this task should be a simple matter of importing the gadget we made (See example) on the wiki.

Unfortunately Google does not support Javanese for its OCR service (also see T140037#2521503). We ran into this issue with an Indic language as well.

Thank you; however, we would not use Google API or something similar to do OCR service for Javanese as far as I know.

The software developed by the university has its own system. I have not yet tested the software nor do I have its documentation at the moment. But I would love to request if anyone is interested.

Unfortunately Google does not support Javanese for its OCR service (also see T140037#2521503). We ran into this issue with an Indic language as well.

Thank you; however, we would not use Google API or something similar to do OCR service for Javanese as far as I know.

The software developed by the university has its own system. I have not yet tested the software nor do I have its documentation at the moment. But I would love to request if anyone is interested.

Oh, that's interesting. Do you know if the service has an API that can be used to fetch the OCR text?
The code does not necessarily have to live on the Wikimedia site. We can use the API for the service (If it is running somewhere outside) and use a gadget to automatically paste the OCR text in the editor. This is similar to how the gadget I mentioned above works.

@Rachmat04: Do you know if the service has an API that can be used to fetch the OCR text?

Fexpr added a comment.Mar 17 2019, 3:32 PM

@Niharika @Aklapper Hi! I am sorry for the lack of updates regarding this project. The process of annotating the scriptures was very slow, so for the last three months we have been working on increasing the labelled data by crowdsourcing the process. Now that the training data is ready, the prototype is ready, we're back on track on developing the software and I'd like to catch up on the issue of integrating the software so it can be used with Wikisource.

Since we are still making improvements here and there with the software, we decided not to put it directly in the WMF Labs/ Wikimedia site for now, we will put the software on our own server. For your information, the program is built by using Python 3 and PHP, and the database for the annotated label is in SQL (we are planning to make it to XML in the later phase so that it can be used easily by everyone outside this project).

The program will take images as an input, and it will give plain text as the output. Would you please explain to me how I can fetch the images from each page which is opened by a user in Wikisource (so that it can be used by our software)? Is there any documentations about this that I can read?

As for the API to fetch the OCR text to Wikisource, we haven't build it yet, so for now it just give a display of the OCR to the user and the user has to copy-paste the result by themselves to Wikisource interface. Do you have any ideas for the best practices to implement this? Do we need to put authorization layer of Wiki account so that the OCR text can be fetched directly to the Wikisource interface?

Thank you very much for your help!

Would you please explain to me how I can fetch the images from each page which is opened by a user in Wikisource (so that it can be used by our software)? Is there any documentations about this that I can read?

Sounds to me like a gadget (list of gadgets on English Wikisource) could be written that adds a link to the sidebar (or such) on Wikisource to allow the user to send the image to your tool? In my understanding running this on your own server instead of using Toolforge or CloudVPS could make you run into CORS issues and potential privacy policy issues. A developer may please correct my half-knowledge if I am wrong here.

Do you have any ideas for the best practices to implement this?

Does the service has an API that can be used to fetch the OCR text?

Do we need to put authorization layer of Wiki account so that the OCR text can be fetched directly to the Wikisource interface?

Not sure if I understand correctly... OAuth might be related here?

dwika added a subscriber: dwika.EditedMay 14 2019, 6:01 AM

Does the service has an API that can be used to fetch the OCR text?

We are developing the API for it, but have no idea on how to connect to the Wiki server.
Is there any guidance for sending and requesting data to and from Wiki server? Do we need to have specific account?
I would be very grateful for your information on having connection to Wikiserver.

Aklapper added a comment.EditedJul 2 2019, 10:01 PM

Is there any guidance for sending and requesting data to and from Wiki server? Do we need to have specific account?

@dwika: I'm sorry for my belated answer. Please see https://www.mediawiki.org/wiki/API:Main_page for how to use the MediaWiki API.
If you shared more technical details it would be easier to be more specific and supportive...
Please see https://discourse-mediawiki.wmflabs.org/ for general development questions.

Aklapper added a comment.EditedJul 27 2019, 9:09 PM

@Rachmat04 / @dwika: Any specific problems that we can help with / technical details that you could share?

MJL moved this task from Backlog to Backlog (Proofreader) on the Wikisource board.Sep 9 2019, 11:57 PM

@Rachmat04 / @dwika: Any news to share? Is any of you working on this? If yes, do you need any help?

dwika added a comment.May 30 2020, 2:19 AM

Our problem is how to save a UI code in WikiSource server? The engine runs well on a rented server and I guess the principle workflow is the same only running on different server. The problem is that we do not know the address of WikiSource server, or an access to put (write/save) our UI (API) code to that server. It seems trivial but it turns out complicated if we have no access.

Our engine called Cakra OCR has been running in our experimental server https://trawaca.id/ocrjawa/?lang=en with a working example shown below:


An API has also been set up, although we are still fixing bits of it. We've tested the API from an external server, for example from our friend's https://ferianto.id/twclient/sampleajax.php and it worked in receiving JPEG images of Javanese manuscript and returning the OCR result in Unicode.
We've stuck in how to make interface like this within the Wikisource:

where window A will send the material through Cakra API and the resulting OCR will be send into window B, just like in our server.

Thanks for the update. https://wikitech.wikimedia.org/wiki/Portal:Toolforge explains how you could host this on Wikimedia Toolforge, if the source code of the Cakra OCR engine is open source. Could you provide a link to the source code of the Cakra OCR engine?

Xover added a subscriber: Xover.May 30 2020, 11:50 AM

We've stuck in how to make interface like this within the Wikisource:


where window A will send the material through Cakra API and the resulting OCR will be send into window B, just like in our server.

@Mahastama There are typically two components to this kind of OCR system: 1) the OCR engine itself, hosted on a server somewhere and providing an API that can be called over the web, and 2) a javascript triggered by a button in the Wikisource editor window, that sends the URL of the page image to the OCR server and gets the extracted text back.

For the server component, if you don't already have a suitable server or don't want to host such a server, you can ask for web hosting on Toolforge (see the link Aklapper provided above). Toolforge is a semi-official shared hosting facility for various tools used by volunteers on Wikimedia projects.

For the interface that's visible to end users, this will consist of a javascript triggered by a button in the text editing window. Such javascripts can be stored in several locations on Mediawiki wikis, for different purposes and with different ramifications for necessary permissions, ease of use, policy, and so forth. For getting started testing you can just make a script as a sub-page of your user page on the wiki. For example at https://id.wikisource.org/wiki/Pengguna:Mahastama/OCR.js (or another sensible name).

Mediawiki automatically runs javacript statements placed in various pages, some globally for the whole wiki and some just for your user account (and some only when you use a specific user interface skin). Once you have the script you can tell Mediawiki to load it for your user account by putting importScript('Pengguna:Mahastama/OCR.js'); into a subpage named common.js in your user account (i.e. at https://id.wikisource.org/wiki/Pengguna:Mahastama/common.js). Mediawiki will load your common.js, and it will in turn load your OCR.js script.

The "OCR.js" script will contain code to add a button to the wiki editor toolbar; attach an event handler that will call a function in your script to that button; to extract the URL of the page image from the current page; send that URL to your OCR server; receive the returned text; and replace the contents of the editing box with that text. You can see an example of a script that does this at https://wikisource.org/wiki/MediaWiki:GoogleOCR.js.

Once you have something that works there are various ways to make it available for other users. One is to just have all other users manually load the script the same way you did (other users can load the script in your user page but can't modify it). The easiest for end users to use method is to turn it into a "Gadget", because gadgets can show up in their preferences as a simple checkbox to enable. This has some policy implications for the wiki on which it is going to be used, and some for our global privacy policy, so you need to be prepared for some discussion etc. at that point.

PS. Community-Tech will be working on a related system this year (not specifically for Javanese, but an OCR system of the same general shape), so it is possible they will want to discuss opportunities for collaboration with you. I believe their idea is to make an OCR utility that can use multiple backends, and it is possible your system could be one of those backends. If that's the case you may not need to worry about making any separate front end / user interface for this.

Tpt added a subscriber: Tpt.May 31 2020, 8:14 AM

@Mahastama I have been bold and just created a beginning of script here: https://id.wikisource.org/wiki/Pengguna:Mahastama/OCR.js
I have put it in one of your user subpage to allow you to edit it. I hope it is fine for you.
It adds an "OCR" button to the page pages and calls the Trawaca API just like you presented.
However, currently the Trawaca API fails with an "authorization" error.
May you or one of the OCR developer have a look at it?
To reproduce it, you have to load the script (just like explained by @Xover, and try to run the OCR on any Page: page.

Thanks guys for the enlightment @Aklapper, @Xover and @Tpt :) Me and my team have been trying for months to find the way, because we are unfamiliar with the Wikisource site structure (nobody ever worked using this before). Many thanks for @Tpt for making a working example; that's exactly what we need. Sorry for the "authorization" error, as one of my developer happened to change the API key, I'll take a look at it.
Well, if the tech community is considering to make a common interface for calling external OCR API's, we'll be glad to collaborate.

Thanks guys, it's been up and running for the initial test. Apparently the module which we use to accept cross-domain API request has a very short valid period for each API key generated and caused the error mentioned before. We temporarily disable the validity period check at the moment.


The textbox is already showing the response from the Cakra OCR API. However there are some things still in question:

  1. We haven't tested it yet using a Wikisource document with Javanese script. I am asking the ID crew to point an example of available file.
  2. The sent data from the page is apparently in BLOB data type, while the API is designed to receive a JPEG file. We might have to take a look at the process and the data sent, if this will raise an issue on our side or not.
  3. The OCR result in Unicode jv is contained inside the span tag. Do we have to provide a response in plain text or is it possible to circumvent this so that only the result from inside the span tag is displayed on the textbox? The blank return itself is still within investigation, on whether it is caused by no Javanese script in the document or because the sent data is a BLOB.