Page MenuHomePhabricator

Update translation-server, citoid
Closed, DeclinedPublic0 Story Points

Tokens
"Love" token, awarded by Sebastian_Berlin-WMSE."Meh!" token, awarded by Lokal_Profil."Love" token, awarded by MichaelSchoenitzer."Like" token, awarded by MichaelSchoenitzer_WMDE."Burninate" token, awarded by czar."Cookie" token, awarded by Mvolz.
Assigned To
None
Authored By
Mvolz, Jul 16 2016

Description

Translation-server has not been updated in two years, and there are much desired updates to it, see: https://github.com/zotero/translation-server/issues/31, https://github.com/zotero/translation-server/issues/19

Note that the most recent version of translation-server now requires an additional set-up command: ./fetch_sdk https://github.com/zotero/translation-server

Event Timeline

Mvolz created this task.Jul 16 2016, 3:58 PM
Restricted Application added a project: VisualEditor. · View Herald TranscriptJul 16 2016, 3:58 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald Transcript
Mvolz triaged this task as Normal priority.Jul 16 2016, 3:58 PM
Mvolz updated the task description. (Show Details)Jul 16 2016, 4:06 PM
Mvolz updated the task description. (Show Details)Jul 16 2016, 4:12 PM
mobrovac removed mobrovac as the assignee of this task.Jul 18 2016, 7:35 AM
mobrovac added subscribers: akosiaris, mobrovac.

xulrunner cannot be updated, it hasn't been packaged for a while. @akosiaris could you update zotero/translation-server perhaps?

Jdforrester-WMF set the point value for this task to 0.Jul 19 2016, 7:02 PM
Jdforrester-WMF moved this task from To Triage to TR0: Interrupt on the VisualEditor board.
Mvolz moved this task from Backlog to Service on the Citoid board.Jul 29 2016, 1:01 PM

I somehow missed this. @Mvolz approached me today, I 'll help update the zotero/translation-server.

Updating xpcshell/xulrunner is impossible though as @mobrovac pointed out.

Mvolz updated the task description. (Show Details)Aug 1 2016, 12:54 PM
Mvolz updated the task description. (Show Details)
Mvolz updated the task description. (Show Details)Aug 1 2016, 12:56 PM
Mvolz renamed this task from Update translation-server, xpcshell, citoid to Update translation-server, citoid.Sep 19 2016, 1:05 PM
Mvolz updated the task description. (Show Details)
Mvolz updated the task description. (Show Details)Sep 19 2016, 4:32 PM

FWIW, I have exactly 0 time to help with this update. Hope to do so soon.

Mvolz moved this task from Service to Production on the Citoid board.Oct 28 2016, 3:17 PM

Change 330684 had a related patch set uploaded (by Alexandros Kosiaris):
WIP: Update the zotero translation-server deploy repo

https://gerrit.wikimedia.org/r/330684

Mvolz awarded a token.Jan 12 2017, 9:41 AM
czar added a subscriber: czar.Mar 15 2017, 5:45 AM
czar updated the task description. (Show Details)Mar 30 2017, 12:32 PM
Mvolz updated the task description. (Show Details)Mar 30 2017, 12:34 PM

@czar, that's covered under the first task, which is to update to latest version (which includes the ability to use em).

czar awarded a token.Apr 3 2017, 7:05 PM

I am wondering about the status of this issue. It seems still open, but on the other hand the GitHub ticket https://github.com/zotero/translators/issues/1088 (and the related ones) is closed since months. The linked changes here https://gerrit.wikimedia.org/r/330684 seems to solve (part of?) the tasks.

Members of the German Wikipedia did some time ago some changes to upstream, to fix several problems the German community is facing. So people from the German Wikipedia Community wait for this to happen.

Mvolz updated the task description. (Show Details)Dec 13 2017, 2:13 PM
Mvolz mentioned this in T121992: Zotero OOM.
mark added a subscriber: mark.Jan 8 2018, 3:25 PM
Lokal_Profil added a subscriber: Lokal_Profil.

This is now blocking new translators and the Zotero translator admins have made it clear that they will not allow backwards compatible translators to be added to the repo just for the sake of Wikimedia.

It is not just about any new Zotero translators but actually the "includes" function is also used in many of the existing translators now. Currently, there are 43 Zotero translators including the "includes" function, and will therefore not work in their newest version anymore with Citoid:

$ grep '.includes(' *.js -l
ACLWeb.js
ACM Digital Library.js
Air University Journals.js
Antikvarium.hu.js
Archeion.js
AustLII and NZLII.js
CiteSeer.js
CiteULike.js
Dialnet.js
Eastview.js
El Pais.js
Erudit.js
FAO Publications.js
FreePatentsOnline.js
Google Patents.js
Idref.js
jmlr.js
KStudy.js
Le Monde.js
Library Catalog (Encore).js
Library Catalog (PICA).js
Max Planck Institute for the History of Science Virtual Laboratory Library.js
MIT Press Journals.js
Musee du Louvre.js
NASA ADS.js
National Library of Australia (new catalog).js
National Library of Norway.js
Nature Publishing Group.js
Oxford Reference.js
Primo Normalized XML.js
Probing the Past.js
ProQuest Ebook Central.js
Reuters.js
R-Packages.js
SAGE Journals.js
ScienceDirect.js
Semantic Scholar.js
Springer Books.js
Sveriges radio.js
The Economist.js
Thieme.js
Twitter.js
Zenodo.js
This comment was removed by Zuphilip.

As an FYI, T187194 has been filled in February under the is https://www.mediawiki.org/wiki/Code_stewardship_reviews context. At this point in time it remains unclear if and when an upgrade can/will happen.

@Lokal_Profil and I discussed how we can move forward with making new translators and our proposal is:

  1. Write translators for the latest version of Zotero, add them to Zotero's translator repo and sync Wikimedia's translator repo, as usual.
  2. Push patch to Wikimedia's translator repo with code that is compatible with the version of translation-server that Citoid is using.
  3. Future step: once Wikimedia's translation-server is updated, revert patches for older version.

This way, it's still possible to contribute to Zotero and create translators that will work on Wikipedia. The drawback is that there will be a bit of extra work, to make the translators compatible with the old translation-server, but unless more issues are discovered, it should be manageable.

Would this be a reasonable solution from Wikimedia's side?

@Lokal_Profil and I discussed how we can move forward with making new translators and our proposal is:

  1. Write translators for the latest version of Zotero, add them to Zotero's translator repo and sync Wikimedia's translator repo, as usual.
  2. Push patch to Wikimedia's translator repo with code that is compatible with the version of translation-server that Citoid is using.
  3. Future step: once Wikimedia's translation-server is updated, revert patches for older version.

There is an implicit assumption here. That translation-server will be updated. Depending on outcome of T187194 that might or might not happen. If the former happens, cool, but if it's the latter the service will have to discontinued and removed from the infrastructure.

This way, it's still possible to contribute to Zotero and create translators that will work on Wikipedia. The drawback is that there will be a bit of extra work, to make the translators compatible with the old translation-server, but unless more issues are discovered, it should be manageable.

Would this be a reasonable solution from Wikimedia's side?

It would certainly provide requested functionality that is currently missing from the infrastructure but keep in mind that if the result of T187194 is negative it will be wasted work.

Mvolz added a comment.Mar 29 2018, 9:24 AM

@Lokal_Profil and I discussed how we can move forward with making new translators and our proposal is:

  1. Write translators for the latest version of Zotero, add them to Zotero's translator repo and sync Wikimedia's translator repo, as usual.
  2. Push patch to Wikimedia's translator repo with code that is compatible with the version of translation-server that Citoid is using.
  3. Future step: once Wikimedia's translation-server is updated, revert patches for older version.

    This way, it's still possible to contribute to Zotero and create translators that will work on Wikipedia. The drawback is that there will be a bit of extra work, to make the translators compatible with the old translation-server, but unless more issues are discovered, it should be manageable.

    Would this be a reasonable solution from Wikimedia's side?

I'd be happy to merge such a patch unless @mobrovac has any objections.

Yes, it will make future merges a PITA but that's already the case, as translators don't automatically merge anyway in git (the "last updated" string causes merge conflicts every time a translator is updated.)

@Lokal_Profil and I discussed how we can move forward with making new translators and our proposal is:

This way, it's still possible to contribute to Zotero and create translators that will work on Wikipedia. The drawback is that there will be a bit of extra work, to make the translators compatible with the old translation-server, but unless more issues are discovered, it should be manageable.

Would this be a reasonable solution from Wikimedia's side?

It would certainly provide requested functionality that is currently missing from the infrastructure but keep in mind that if the result of T187194 is negative it will be wasted work.

"Wasted" is a little strong here. It provides a real and immediate current benefit for however long it's running. And, Zotero is a project that is used in many other contexts so worse comes to worse you're helping the open source community at large ;).

"Wasted" is a little strong here. It provides a real and immediate current benefit for however long it's running. And, Zotero is a project that is used in many other contexts so worse comes to worse you're helping the open source community at large ;).

Yes, "Wasted" might indeed a bit strong. I do see your point, so to clarify, what I was trying to communicate (and in order to avoid unwarranted expectations that might lead to frustration) is that it's possible that the resulting work will not be used in the wikimedia infrastructure/context, at least in the long term.

To 2: You can also add an implementation of the "includes" function to every translator code, such that patching can be done automatically, e.g. https://stackoverflow.com/questions/31221341/ie-does-not-support-includes-method#31221374 .

The proposed approach has immediate gratification, but I would be reserved if any serious work would need to be done on that front (regardless of the outcome of T187194). Realistically, continuing to support Zotero in its current form in our production environment is less and less likely. Personally, I think that it would be better to do some serious work towards implementing the interfaces directly in Citoid (cf. T93579: Restructure so that citoid can be run without Zotero), allowing us to keep using the translators (for the exact effort needed cf. T93579#2156135).

Mvolz closed this task as Declined.
Mvolz removed a project: Patch-For-Review.

Declined in favour of T197242

Change 330684 abandoned by Mvolz:
WIP: Update the zotero translation-server deploy repo

Reason:
abandon in favour of translation-server-v2 https://phabricator.wikimedia.org/T197242

https://gerrit.wikimedia.org/r/330684