Write a Zotero translator and document process for creating new Zotero translator and getting it live in production
Closed, ResolvedPublic8 Story Points

Description

Citoid is a service that allows people to easily add references on Wikipedias. It relies heavily on Zotero's translation-server, which in turn uses various translators to get citation metadata from specific pages.

Sebastian from Zotero did a tech talk on how to do this: https://www.mediawiki.org/wiki/Citoid/Zotero%27s_Tech_Talk

However it would be nice to have a page on mediawiki.org that details how to do this with citoid specifically in mind.

Zotero has their own documentation on how to do this: https://www.zotero.org/support/dev/how_to_write_a_zotero_translator_plusplus

However, this documentation is only available for writing translators for the browser plug-in, not translation-server. Translators often work both in the browser and translation-server, but for citoid's purposes it is necessary for any new translators to work in translation-server, so more targeted documentation specifically for citoid would be nice. If relevant, adding/fixing documention on Zotero's wiki as well as on mediawiki.org can be part of the scope of the project.

Writing a new translator for Zotero's translation-server is also within scope of the project as part of the process of documenting it. Translators are in Javascript. Familiarity with HTML is helpful as translators are essentially web scrapers. (Review of this component would be done be Zotero programmers, not WMF staff.) There are many requested translators on the translator repo so there are lots to choose from: https://github.com/zotero/translators/labels/New%20Translator

Relevant bugs to this task: T94170, T94663

Getting started

Internship project details

Related Objects

There are a very large number of changes, so older changes are hidden. Show Older Changes

Hey @Mvolz, this project seems really interesting and I'm willing to take up this for Outreachy-13. Can you help me how to get started?

Hey @Mvolz, this project seems really interesting and I'm willing to take up this for Outreachy-13. Can you help me how to get started?

Please read through https://wiki.gnome.org/Outreachy#Eligibility and see the eligibility criteria for Outreachy.

Hii @Mvolz @czar @LuisVilla , this project seems really interesting and I'd really like to take this up. Can you guide me on how to get started?

Hii @Mvolz @czar @LuisVilla , this project seems really interesting and I'd really like to take this up. Can you guide me on how to get started?

Hi @Soum213! Thanks for your interest! Please see https://www.mediawiki.org/wiki/How_to_become_a_MediaWiki_hacker#Feedback.2C_questions_and_support for more general information and how to get started.

Hello,
I went through the documentation about Zotero, and even made a translator halfway (using the browser plug-in). I have experience in (non-)technical editing, so I think I will be comfortable with documenting the whole process. I am in the final year of college but I have my college vacations from Dec to mid Feb i.e. approximately 10-11 weeks of vacations that coincide with the Outreachy internship period, so I can easily give 40+ hours to it.
I have started working on the microtask T95128, but I needed some direction, can you please guide me with that?

Mvolz added a comment.Oct 9 2016, 4:44 PM

Hello,
I went through the documentation about Zotero, and even made a translator halfway (using the browser plug-in). I have experience in (non-)technical editing, so I think I will be comfortable with documenting the whole process. I am in the final year of college but I have my college vacations from Dec to mid Feb i.e. approximately 10-11 weeks of vacations that coincide with the Outreachy internship period, so I can easily give 40+ hours to it.
I have started working on the microtask T95128, but I needed some direction, can you please guide me with that?

That wasn't very clear, sorry! Those first two bugs were bugs that were reported to us, but it is actually a bug with a Zotero translator, so you should report the bug to https://github.com/zotero/translators and then note in the bug report here what you have done.

Mvolz updated the task description. (Show Details)Oct 10 2016, 2:42 PM
Mvolz updated the task description. (Show Details)Oct 16 2016, 9:06 AM
Mvolz moved this task from Backlog to Zotero & External on the Citoid board.Oct 28 2016, 3:19 PM

@Mvolz @czar @LuisVilla hello! for the very first time, we are considering to submit Wikimedia's application for the Rails Girls Summer of Code program: http://railsgirlssummerofcode.org. It is like Outreachy, and has the same goal of fostering diversity in Open Source. Though it sounds from the program's name that the projects submitted should be ROR/ Rails but that's not true. Projects could be any programming language. I'm wondering what's the current status of this task, and if anyone of you would be interested in mentoring? A faster reply would be much appreciated, as we are pretty close to the deadline (less than 6 days remaining!) :) Thank you!

@srishakatux as far as I know, this is still a basically undocumented process, so I still think it'd be a good project for someone.

(To be clear, though, I can't mentor - I can only test/proof/edit the resulting document.)

czar added a comment.Jan 26 2017, 11:06 AM

@srishakatux Cool! With current commitments, I won't be able to give more than cursory support, but it'll move off the backburner eventually so keep me apprised

@czar @LuisVilla thank you so much! @Mvolz over to you now :)

This comment was removed by srishakatux.

@srishakatux I'm mentoring RGSOC this year for another project, but I'm still awaiting for confirmation whether they accept our projects or not, otherwise Let me know the criteria for mentoring Wikimedia's projects in RGSOC.

@Amit_gkp thanks for showing your interest in mentoring for RGSOC! To be a mentor, you need to be familiar with atleast one of our project's codebase. Could you share which Wikimedia projects are you currently involved with?

@Mvolz and team: Very soon we will be submitting Wikimedia's application for GSOC 2017 and Outreachy round 14, wondering if you would be interested in mentoring this project for either of those?

@srishakatux Last year I did little bit of work on T118463 and here you can spot my commits . Currently, I'm working on Wikidata Tasks like T93375 and T65632.

@Amit_gkp we are looking for mentors who have more experience with our codebase! here are a few example areas in which you can contribute: https://www.mediawiki.org/wiki/How_to_become_a_MediaWiki_hacker. thanks!

SunakshiT added a subscriber: SunakshiT.EditedFeb 17 2017, 5:18 AM

Hii Developers
I am Sunakshi. And I am willing to participate in the Outreachy Round 14. The Zotero translator-server is successfully build in my system. and is running on port:1969. I want to know what are the easy-pick issues I can work on (since the issues under the zotero/translator-server are not labeled)in order to contribute :-)
Thank you.

Johan updated the task description. (Show Details)Feb 17 2017, 12:56 PM

Hi,

I also interested in this project in oureachy 14. What is the good task for newcomer? Could I just pick up any issue in Zotero translator project, that labeled as easy?

Kate

Mvolz updated the task description. (Show Details)Feb 21 2017, 4:57 PM
Mvolz updated the task description. (Show Details)
Mvolz updated the task description. (Show Details)Feb 21 2017, 5:06 PM
Mvolz updated the task description. (Show Details)
Mvolz updated the task description. (Show Details)
Mvolz added a comment.EditedFeb 21 2017, 5:12 PM

@katejimme @SunakshiT Thanks for your interest! I have updated the list of microtasks. There are currently two now, I will look for some more.

@Mvolz perhaps one of the translator requests from zotero upstream would be a good microtask? May not be small enough.

(Somewhere around there is a bug listing the most-commonly-cited websites that don't have translators; that might also be a good starting place.)

Hello, I would like to introduce myself. I am Sonali and am also interested in taking this project in outreachy round-14.

Mvolz updated the task description. (Show Details)Feb 26 2017, 3:25 PM

@Mvolz For the microtask T95128: OpenLibrary citation has incorrect author information (bug filed upstream) I have filed an issue on the Zetero/translator upstream.
I edited the task description to include the link of the issue and changed the title from "file bug" to "bug filed".
I haven't changed the status of the task, it still remains open.

Hey @Mvolz @czar
I am Vrinda and I would like to apply for this project in Outreachy round 14. Could you give some idea for a micro task to get acquainted with the repository?

Mvolz added a comment.Feb 26 2017, 9:28 PM

Hmm, as this is a documentation one, I don't think it's eligible for GSoC, only Outreachy? Removing from GSoC... feel free to add back if incorrect.

Is there an IRC channel for communications specific to outreachy/GSoC or we need to communicate on phabricator itself?

I don't think there's a specific one, but for this project,
#mediawiki-services on IRCnode is appropriate.

I think it is #wikimedia-services, I didn't find #mediawiki-services on irccloud
Please let me know if I am mistaken.

Mvolz added a comment.Feb 28 2017, 5:16 PM

Whoops, yup it's #wikimedia-services.

@Mine0901 hi, you are welcome to ask any program-specific queries in #wikimedia-devrel. See here: https://www.mediawiki.org/wiki/Google_Summer_of_Code_2017#Contact

Hey @Mvolz @czar
I would like to know if there are any microtasks available for this issue.

Mvolz updated the task description. (Show Details)Mar 6 2017, 12:29 PM
Mvolz updated the task description. (Show Details)
Mvolz updated the task description. (Show Details)Mar 6 2017, 12:34 PM

@SunakshiT I have added two more microtasks to the description.

I deleted the outdated translator and submitted a PR https://github.com/zotero/translators/pull/1250

czar updated the task description. (Show Details)Mar 12 2017, 8:52 AM
czar updated the task description. (Show Details)
Mvolz added a comment.Mar 12 2017, 9:11 AM

@czar, the microtasks were for applicants to do. :)

Mvolz added a comment.Mar 12 2017, 9:12 AM

@czar, the microtasks were for applicants to do. :)

czar updated the task description. (Show Details)Mar 12 2017, 9:12 AM
Restricted Application added a subscriber: Sadads. · View Herald TranscriptMar 12 2017, 9:12 AM
Mvolz added a comment.Mar 12 2017, 9:13 AM

We still have two more weeks until the application deadline so we should perhaps find a few more and leave them for other people to do so everyone has a fair shot at applying.

czar added a comment.EditedMar 12 2017, 9:21 AM

@Mvolz, I was just perusing through the Github issues—didn't even know/realize they were also microtasks on Phabricator!

How small are microtasks supposed to be? I think it would be a good introductory exercise to write a translator using Framework (FW). Relatively easy to bang these out—for example, here's one I did today for frieze.com—get acquainted with XPath, basic Zotero.Utilities functions, follow a template, etc. Then someone could move on to those mid-level translators which should be coded without Framework (FW) from the ground up.

czar added a comment.Mar 12 2017, 9:23 AM

Concerning the tutorial to be written (on how to write translators): any suggestions for the website to be used as the example? Ideally it would be a site available under a free use license such that we can take screenshots of its design and code.

czar updated the task description. (Show Details)Mar 12 2017, 9:45 AM
czar updated the task description. (Show Details)
czar updated the task description. (Show Details)Mar 12 2017, 10:15 AM

Concerning the tutorial to be written (on how to write translators): any suggestions for the website to be used as the example? Ideally it would be a site available under a free use license such that we can take screenshots of its design and code.

I believe http://www.example.com is designed for a similar purpose. Although it doesn't have very rich metadata, it would probably be fine for this. Or we could always use a WMF site!

@czar I am learning to write a translator. I have some questions related to Scaffold and about the use of framework.js
Since I couldn't find you on IRC, Can you provide a medium to talk?

czar added a comment.Mar 16 2017, 7:32 PM

Sent a Phabricator chat invite

I should note that the maintainers are looking to possibly deprecate Framework (FW). I'm not wedded to FW in specific, but I think it's slightly easier than writing a full translator. Perhaps some others might want to chime in.

For examples of other translators in FW, I have a dozen recent pull requests in waiting: https://github.com/zotero/translators/pulls

Mine0901 updated the task description. (Show Details)Apr 25 2017, 1:51 PM
Tbayer added a comment.Sep 6 2017, 9:06 PM

According to today's blog post, this project has now concluded. I assume this is the outcome? https://www.mediawiki.org/wiki/Citoid/Creating_Zotero_translators
Thanks for your work on this important topic!

Elitre added a comment.Sep 7 2017, 4:24 PM

What he said! TYVM!!!

Hello! Thank you for featuring this project in the previous edition of Outreachy. Did the student complete the project requirements? In any case, please help modify the task description and add/remove the tags accordingly. Thank you!

Mvolz closed this task as Resolved.Jun 14 2018, 3:43 PM
Mvolz assigned this task to Mine0901.
Restricted Application added a project: User-Ryasmeen. · View Herald TranscriptJun 14 2018, 3:43 PM