Create Zotero translator for sverigesradio.se
Closed, ResolvedPublic4 Story Points

Description

Create a Zotero translator for http://sverigesradio.se/. Documentation for translators: https://www.zotero.org/support/dev/translators. As the first translator task, this will also include setting up everything Zotero related development.

Jopparn changed the point value for this task from 16 to 4.Feb 19 2018, 8:57 AM

The translator itself may be done now. It handles things like:

  • Creator - in some cases the article has no author; the byline just says "Ekot", e.g. https://sverigesradio.se/sida/artikel.aspx?programid=83&artikel=6821850
  • Date - handles three different cases, depending on how long ago the article was published; articles published the current year only have day and month, and articles published the current day only has time.
  • Title - retrieved from <h1> to not include superfluous " - Nyheter (Ekot)".

What remains is to figure out if all information is included and how to get it to work with Citoid.

I've also taken a quick look to se if there is an easy way of determining which pages link to sverigesradio.se without embedding the link in a webbref/tidningsref template. However since text content isn't contained in the replicas quary.wikimedia.org cannot be used for this. One can however use the built in search on Wikipedia which yields ~1300 hits on on svwiki and indicates that there are more on other wikis.

This query could be of use when advertising that this translator has been deployed.

The pull request above has now been merged.

The pull request above has now been merged.

Great. Can you request it to be pulled into the WMF repo?

Even though this should be working with T188893 done, automatically generated references aren't formatted properly on SVWP. I'll need to investigate where the problem lies.

The problem has (hopefully) been found and a fix is in the process of getting deployed. See T188893#4083624 for details.

The people at Zotero aren't willing to merge the changes that would add backwards compatibility. While understandable, this unfortunately means that the translator won't work with Citoid.

Possible solutions:

  1. Wait for Citoid's translation-server to get updated. Judging by T140539, this won't be any time soon.
  2. Push changes directly to the Wikimedia translator repo, if this is possible. Sadly, this means we won't be contributing to Zotero.

There's also the matter of the remaining translators: T187024, T187025 and T187026. Most of the work is done for these, but if they won't work on SVWP, is it worth it to even push the to Zotero?

Sebastian_Berlin-WMSE changed the point value for this task from 4 to 1.Apr 3 2018, 6:12 AM
Sebastian_Berlin-WMSE changed the point value for this task from 1 to 2.

Change 423675 had a related patch set uploaded (by Sebastian Berlin (WMSE); owner: Sebastian Berlin (WMSE)):
[mediawiki/services/zotero/translators@master] Update translator for Sveriges Radio to make it compatible with Citoid

https://gerrit.wikimedia.org/r/423675

The patch above introduces the polyfill solution proposed in T140539#4091055.

@Mvolz, would you be willing to take a look at https://gerrit.wikimedia.org/r/423675? It would be great to get this (hopefully) last issue fixed.

@Mvolz, would you be willing to take a look at https://gerrit.wikimedia.org/r/423675? It would be great to get this (hopefully) last issue fixed.

Sorry, was on holiday for Easter.

Looks good - although I wonder if maybe we should insert the polyfill for all of the translators and then save that as a patch somewhere as one big commit and just re-apply it after every update - or maybe have a little script that does it for all of them -and then undo the commit before every update. @mobrovac thoughts?

I'd second having the polyfill added in general. Perhaps we could put it in a separate file and then include it in all of the translators? IMO, that way we wouldn't have conflicts when rebasing on top of upstream.

Mvolz added a comment.May 3 2018, 12:29 PM

I'd second having the polyfill added in general. Perhaps we could put it in a separate file and then include it in all of the translators? IMO, that way we wouldn't have conflicts when rebasing on top of upstream.

I'm not sure this will work, at least not easily; translator files are treated specially by Zotero. For instance if you put any non-translator files in the directory it chokes (because it expects the translator JSON block at the top of every file).

Sebastian_Berlin-WMSE claimed this task.
Sebastian_Berlin-WMSE changed the point value for this task from 2 to 4.

@Sebastian_Berlin-WMSE, check what versions you run, per @Mvolz comment in 423675.

@Mvolz, has there been any progress on this?

Is there anything I can do to help this moving forward? I'd like to get it done before too long.

Mvolz added a comment.Jun 18 2018, 9:19 AM

Is there anything I can do to help this moving forward? I'd like to get it done before too long.

No, sorry! I feel bad this has been left in limbo so long, but we were trying to sort out infrastructure. It looks like we're going to be deploying translation-server-v2 when it's done (T197242) and then your translator will be compatible - I think waiting for that might be the best bet? It's not ideal, I know.

Sebastian_Berlin-WMSE changed the task status from Open to Stalled.Jun 18 2018, 10:30 AM

Thanks for the update, @Mvolz. I wasn't subscribed to T187194, so I didn't see the development there. Now I should be more in the loop.

I agree that it makes sense to wait for T197242 rather than to fix the issues with the current version of the server.

Change 423675 merged by Mobrovac:
[mediawiki/services/zotero/translators@master] Update translator for Sveriges Radio to make it compatible with Citoid

https://gerrit.wikimedia.org/r/423675

With the merge above, would the new translator be running? I tried adding a citation and still get the old result, but maybe T197242 is still required.

Mentioned in SAL (#wikimedia-operations) [2018-07-05T09:40:49Z] <mobrovac@deploy1001> Started deploy [zotero/translators@d6702bc]: String.includes() polyfill for Sveriges radio translator - T187023

Mentioned in SAL (#wikimedia-operations) [2018-07-05T09:40:57Z] <mobrovac@deploy1001> Finished deploy [zotero/translators@d6702bc]: String.includes() polyfill for Sveriges radio translator - T187023 (duration: 00m 08s)

mobrovac changed the task status from Stalled to Open.Jul 5 2018, 9:42 AM
mobrovac triaged this task as Normal priority.

With the merge above, would the new translator be running? I tried adding a citation and still get the old result, but maybe T197242 is still required.

Sorry, I merged yesterday, but it was a bit late to deploy. Did so now, could you try again?

Sebastian_Berlin-WMSE closed this task as Resolved.Jul 5 2018, 9:44 AM

I did and now it works. Thanks for deploying.