DOIs with unscrapeable pages are not merged with crossref / repository metadata
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	JEumerus
	Jan 26 2019, 4:44 PM

Description

This is an issue when on enwiki you try to use Citoid in Visual Editor on Taylor & Francis such as on doi:10.1080/00288306.1980.10424125:

The expected result would be a journal citation similar to {{cite journal |last1=Brothers |first1=R. N. |last2=Heming |first2=R. F. |last3=Hawke |first3=M. M. |last4=Davey |first4=F. J. |title=Tholeiitic basalt from the Monowai seamount, Tonga-Kermadec ridge (Note) |journal=New Zealand Journal of Geology and Geophysics |date=July 1980 |volume=23 |issue=4 |pages=537–539 |doi=10.1080/00288306.1980.10424125 |ref=harv}}

What it actually produces is {{Cite web|url=https://www.tandfonline.com/action/captchaChallenge?redirectUrl=https%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Fabs%2F10.1080%2F00288306.1980.10424125&|website=www.tandfonline.com|doi=10.1080/00288306.1980.10424125|access-date=2019-01-26}} as if it can't recognize the DOI correctly.

Details

	Subject	Repo	Branch	Lines +/-
	Add test	mediawiki/services/citoid	master	+10 -0
	Use zotero search endpoint for dois	mediawiki/services/citoid	master	+208 -103

Customize query in gerrit

Related Objects

Mentioned In: T215755: Tamu.edu DOI is not correctly recognized by Citoid
T210806: Decreased internationalisation of automatic citations as a result of switch to new translation-server
Mentioned Here: T210806: Decreased internationalisation of automatic citations as a result of switch to new translation-server
T215755: Tamu.edu DOI is not correctly recognized by Citoid
T197242: Transition citoid to use Zotero's translation-server-v2

Event Timeline

JEumerus created this task.Jan 26 2019, 4:44 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 26 2019, 4:44 PM

That's definitely bad. We used to merge results if we had the doi with citoid results but we don't do that with a zotero ones; we should either merge with zotero results, just return the crossref one, or use zotero's search input which I assume handles this better.

Mvolz claimed this task.Jan 26 2019, 8:27 PM

Mvolz triaged this task as High priority.

Change 486861 had a related patch set uploaded (by Mvolz; owner: Mvolz):
[mediawiki/services/citoid@master] [WIP] User zotero search for dois

https://gerrit.wikimedia.org/r/486861

gerritbot added a project: Patch-For-Review.Jan 28 2019, 2:37 PM

Change 486861 merged by jenkins-bot:
[mediawiki/services/citoid@master] Use zotero search endpoint for dois

https://gerrit.wikimedia.org/r/486861

Mvolz renamed this task from Taylor & Francis DOI is not correctly recognized by Citoid to DOIs with unscrapeable pages are not merged with other metadata.Feb 11 2019, 10:08 AM

Mvolz merged a task: T215755: Tamu.edu DOI is not correctly recognized by Citoid.

Mvolz edited projects, added Regression; removed Patch-For-Review.

Mvolz moved this task from Backlog to Waiting on Deploy on the Citoid board.

FYI this is a regression caused by T197242. The issue is that a lot of scraping that used to be done by citoid is now done by zotero, but we were handing zotero just the url and not the doi and so the info from the doi wasn't getting merged with what was scraped on the page.

Mvolz renamed this task from DOIs with unscrapeable pages are not merged with other metadata to DOIs with unscrapeable pages are not merged with crossref / repository metadata.Feb 11 2019, 10:36 AM

Mentioned in SAL (#wikimedia-operations) [2019-02-11T21:09:15Z] <mobrovac@deploy1001> Started deploy [citoid/deploy@0b91bea]: Use Zotero for DOIs and pass it the A-L header - T214766 T210806 T215755

Stashbot mentioned this in T210806: Decreased internationalisation of automatic citations as a result of switch to new translation-server.Feb 11 2019, 9:09 PM

Stashbot mentioned this in T215755: Tamu.edu DOI is not correctly recognized by Citoid.

Mentioned in SAL (#wikimedia-operations) [2019-02-11T21:13:03Z] <mobrovac@deploy1001> Finished deploy [citoid/deploy@0b91bea]: Use Zotero for DOIs and pass it the A-L header - T214766 T210806 T215755 (duration: 03m 47s)

Both DOIs seem to be working in production but it appears I forgot to write a test for this, so I'll re-open until that's merged. :).

Mvolz reopened this task as Open.Feb 15 2019, 5:48 PM

Change 492979 had a related patch set uploaded (by Mvolz; owner: Mvolz):
[mediawiki/services/citoid@master] Add test

https://gerrit.wikimedia.org/r/492979

gerritbot added a project: Patch-For-Review.Feb 26 2019, 9:56 AM

Mvolz closed this task as Resolved.Feb 26 2019, 9:57 AM

Change 492979 merged by jenkins-bot:
[mediawiki/services/citoid@master] Add test