Page MenuHomePhabricator

Support meta tag refresh redirects in citoid to support elsevier's linking hub
Open, NormalPublic

Description

A lot of DOIs point to Elsevier's linking hub, which uses browser based redirects, i.e.

https://linkinghub.elsevier.com/retrieve/pii/S0305049198000224

returns a 200 and the page content include the following meta tag:

<meta HTTP-EQUIV="REFRESH" content="2; url='/retrieve/articleSelectSinglePerm?Redirect=https%3A%2F%2Fwww.sciencedirect.com%2Fscience%2Farticle%2Fpii%2FS0305049198000224%3Fvia%253Dihub&amp;key=07cc720d5d7c8d9451bc53530e33b8df38f1ecce\'"/>

This is an issue for transitioning to translation-server-v2 as it will just scrape the redirect page and return those results.

Event Timeline

Mvolz triaged this task as High priority.Sep 11 2018, 9:31 AM
Mvolz created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 11 2018, 9:31 AM

We've added support for this in translation-server:

https://github.com/zotero/translation-server/commit/aa7179e3ced515ce104b5ad4a666941023da3aa9

(We were in the process of implementing this in the client anyway, where we noticed the same thing.)

Note that ScienceDirect still will fall back to the Embedded Metadata translator due to the translator expecting a JS-rendered page, but that's a separate issue that we're working on. Let us know if you run into any other problems.

Mvolz moved this task from Backlog to Service on the Citoid board.Dec 11 2018, 11:32 AM
Mvolz lowered the priority of this task from High to Normal.Dec 17 2018, 10:26 AM