Page MenuHomePhabricator

Deploy IDS extension to zh.wikisource
Open, Stalled, NormalPublic

Description

This is Shoichi (Zheng-Yi Chang), a board member of Wikimedia Taiwan. I am executing OpenWuDict(a Taiwanese dictionary project) on http://zh.wikisource.org/. Because we need to handle lacking ancient Chinese words, I put the opensource IDS render server on tools.wmflabs.org .

Finally I need the extension, Mediawiki-IDSextension to be put on http://zh.wikisource.org/. This extension add new tag "<ids>" to mediawiki, and it can get lacking ancient Chinese words from the IDS render server. I don't know where to ask, so I ask here. Please help us to resolve it.

Event Timeline

Shoichi created this task.Jun 14 2016, 9:54 AM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptJun 14 2016, 9:54 AM

There will be a few steps involved:

  • Extension needs to be moved into Gerrit
  • Security review of extension

for the major starting points.

Extensions deployed on the production cluster can't rely on labs instances, so that will need to be setup in the production cluster as well.

There will be a few steps involved:

  • Extension needs to be moved into Gerrit

Excuse me,Where is Gerrit?

  • Security review of extension

for the major starting points.
Extensions deployed on the production cluster can't rely on labs instances, so that will need to be setup in the production cluster as well.

Do you mean I also need to put the ids server onto the production cluster? How should I do it?

Excuse me,Where is Gerrit?

https://www.mediawiki.org/wiki/Gerrit

Do you mean I also need to put the ids server onto the production cluster? How should I do it?

Yes, You will need to work with Operations to develop a implementation plan as part of the extension deployment process.

Aklapper renamed this task from Add idsgen extension to wikisource to Deploy idsgen extension to zh.wikisource.Jun 14 2016, 12:49 PM
Aklapper changed the task status from Open to Stalled.
Aklapper triaged this task as Lowest priority.

I've set priority to lowest and status to stalled as this requires numerous steps to be performed before this could be deployed (code security review etc etc).
See the two links provided for a checklist. Thanks!

Alexsh added a subscriber: Alexsh.Jun 14 2016, 1:07 PM

+Subscriber

Dereckson added a subscriber: Dereckson.EditedJun 18 2016, 12:26 AM

@Shoichi so next step is you require a Gerrit repository: https://www.mediawiki.org/wiki/Gerrit/New_repositories#Step_4:_Request_a_Git_repository

(Steps 1 and 2 are already done, read the step 3 content too, but you don't really have the choice as it's targeted for a Wikimedia cluster wiki)

I assume you already have support from zh.wikisource community. If this is not the case, now is probably the good moment to gather such support (that would be frustrating to do all the process to find the community doesn't want the extension).

Meanwhile a Gerrit admin creates the repository, you can start the main documentation page at https://www.mediawiki.org/wiki/Extension:IDSextension

Then, you'll follow https://www.mediawiki.org/wiki/Manual:Extension_registration#Migration_for_extension_developers to use the new extension registration mechanism.

Finally, you'll apply https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Coding_conventions/PHP

Let me know if you're stucked on one of these steps, I can help there.

This should allow the extension to respect coding conventions: https://github.com/MGdesigner/Mediawiki-IDSextension/pull/2

This should allow the extension to respect coding conventions: https://github.com/MGdesigner/Mediawiki-IDSextension/pull/2

I have accepted your pull request thank you. Didn't know the issue before.

I've set priority to lowest and status to stalled as this requires numerous steps to be performed before this could be deployed (code security review etc etc).
See the two links provided for a checklist. Thanks!

OK, I will run these process.

OK.

(Steps 1 and 2 are already done, read the step 3 content too, but you don't really have the choice as it's targeted for a Wikimedia cluster wiki)
I assume you already have support from zh.wikisource community. If this is not the case, now is probably the good moment to gather such support (that would be frustrating to do all the process to find the community doesn't want the extension).

Yes, it's very important. That's what I should do.

Meanwhile a Gerrit admin creates the repository, you can start the main documentation page at https://www.mediawiki.org/wiki/Extension:IDSextension
Then, you'll follow https://www.mediawiki.org/wiki/Manual:Extension_registration#Migration_for_extension_developers to use the new extension registration mechanism.
Finally, you'll apply https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Coding_conventions/PHP
Let me know if you're stucked on one of these steps, I can help there.

Thanks for your help very much. I will go on to next move.

@Shoichi so next step is you require a Gerrit repository: https://www.mediawiki.org/wiki/Gerrit/New_repositories#Step_4:_Request_a_Git_repository
(Steps 1 and 2 are already done, read the step 3 content too, but you don't really have the choice as it's targeted for a Wikimedia cluster wiki)

Excuse me, I think we are at step3,and I have written the extension page on Mediiawiki site.

Originally I just want to deploy it into zh.wikisource. Then yesterday I found on zh.wikipedia ,there is also the same problem of huge lacking Chinese characters ,so I speak about this solution on the forum of zh.wikipedia . Then many people come to discuss it. Many people support it,and I got some very good suggestion which I never thought before. In the discussion , it seems that deploying it to the whole zh wikiXXXXX-sites will be better,so I need to improve the extension better. Then go next step.

What runs on idsgen side?

Could we read somewhere the source code?

Shoichi added a comment.EditedJun 22 2016, 7:22 PM

What runs on idsgen side?
Could we read somewhere the source code?

This one, han3_ji7_tsoo1_kian3_WM. It's an unicode Ideographic Description Sequences renderer. It provide a web api to render an I.D.S as a character image. And the IDS are used to represent a lacking C/J/K/V character ( Or other 2D composited characters writing systems like Maya, ancient Egypt).

I paste some examples here. Considerating not everyone can type Chinese characters,so I provide these examples using English characters and some symbols as "elements",then you can test it by yourself :

⿲☺rz
⿰o⿰rz
⿴囗⿰A⿱HA
⿲◕‿◕

And these new characters will go with normal characters in a mediawiki-site page like this:

And, The java servlet of "han3_ji7_tsoo1_kian3_WM" is this one. If you want to trace it, just from the java source file.

Liuxinyu970226 added a subscriber: Liuxinyu970226.EditedJun 23 2016, 2:49 AM

And, The java servlet of "han3_ji7_tsoo1_kian3_WM" is this one. If you want to trace it, just from the java source file.

Looks like there has a license which is non-free:

本字型檔採用創用CC「姓名標示-禁止改作」3.0臺灣版授權條款釋出。

lit. This font file is licensed under a Creative Commons Attribution-NoDerivs 3.0 Taiwan License.

Liuxinyu970226 renamed this task from Deploy idsgen extension to zh.wikisource to Deploy Ids extension to zh.wikisource.Oct 1 2016, 2:06 AM
awight raised the priority of this task from Lowest to Normal.Oct 19 2016, 10:20 PM
awight added a subscriber: awight.

Prioritizing because it seems there is significant community interest, and the deployment path is reasonable. Feel free to revert or discuss...

I've done some code review of the extension and it's very straightforward. There was an XSS vulnerability, the patch is pending review and merge:
https://github.com/MGdesigner/Mediawiki-IDSextension/pull/5

Getting the extension into Gerrit will be easy, assuming @Shoichi agrees with that plan.

Moving the rendering engine onto the production cluster will be the difficult part, I'll open a subtask for that.

This comment was removed by Shoichi.

Prioritizing because it seems there is significant community interest, and the deployment path is reasonable. Feel free to revert or discuss...
I've done some code review of the extension and it's very straightforward. There was an XSS vulnerability, the patch is pending review and merge:
https://github.com/MGdesigner/Mediawiki-IDSextension/pull/5
Getting the extension into Gerrit will be easy, assuming @Shoichi agrees with that plan.
Moving the rendering engine onto the production cluster will be the difficult part, I'll open a subtask for that.

Sorry, I am busy in October until now (Also was sick for 1 week. ). I agree the plan.

Shoichi added a comment.EditedNov 21 2016, 5:54 AM

And, The java servlet of "han3_ji7_tsoo1_kian3_WM" is this one. If you want to trace it, just from the java source file.

Looks like there has a license which is non-free:

本字型檔採用創用CC「姓名標示-禁止改作」3.0臺灣版授權條款釋出。

lit. This font file is licensed under a Creative Commons Attribution-NoDerivs 3.0 Taiwan License.

About the font, I found that license of font CNS11643 has changed to "Open Government Data License, version 1.0" ,this year. The license is compatible with CC-BY 4.0 International ,so now the font is free.

I have pushed new readme into han3_ji7_tsoo1_kian3 upstream .

zhuyifei1999 updated the task description. (Show Details)Dec 24 2016, 5:22 AM
Aklapper added a subscriber: Shizhao.

@Shizhao: I don't see anything not language-specific about Wikisource in this task. Hence removing the Wikisource tag. If there is, please elaborate.

Err, any security review needed?

Err, any security review needed?

There will be. But as far as I understand it, that's a little far at this point.

This comment was removed by Shoichi.
Samwilson renamed this task from Deploy Ids extension to zh.wikisource to Deploy IDS extension to zh.wikisource.Feb 28 2017, 6:01 AM