enriching wikidata entities with MARC data from libraries
Open, Needs TriagePublic
Actions

Description

We suggest to have a pywikibot script to import marcxml data used by libraries to Wikidata.

MARC is a standard used by many libraries for description of items catalogued by libraries. This includes people, places, institutes etc.

Wikimedia Israel got permission from National Library of Israel to get and use its authority database (xml dump size is about ~2.3GB of text), and we have a starting point for coding with a small subset of the data. We expect that other libraries may want to share their data in similar way in the future.

A scratch for the project: https://github.com/idoivri/marc_to_wikidata
Mapping of Wikidata properties to MARC: https://docs.google.com/spreadsheets/d/1lXxIe1vYFbUaTGUWTFi7Gh9zZzUcKZNSjPCk65qgm2A/edit#gid=0
A subset of the large XML file: https://github.com/eranroz/marc_to_wikidata/blob/master/marcxml_example.xml
A short primer to reading the MarcXML file can be found here: https://docs.google.com/document/d/1BVZZd9cJrAwd_dzfX7abB79dbGrQoQGpJJc_seabaPI/edit

Related Objects

Mentioned In: T135422: Wishlist tasks suggested for the Wikimania Hackathon 2016

Event Timeline

eranroz created this task.Mar 31 2016, 11:23 PM

Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptMar 31 2016, 11:23 PM

Thanks, @eranroz! :)
Awesome stuff...

We are in the Caesarea room, in the north-east part of Hansen building.

NationalLibraryofIsrael updated the task description. (Show Details)Apr 1 2016, 8:51 AM

johl added a subscriber: johl.Apr 1 2016, 8:54 AM

Wikimedia RU currently has a preliminary agreement with the Russian State Library (one of the largest Russian libraries) on the export of MARC database (this is several million records). I hope that in the coming days we will settle the copyright question.

Therefore, such a script would be interesting for us.

In T131417#2167440, @putnik wrote:

Wikimedia RU currently has a preliminary agreement with the Russian State Library (one of the largest Russian libraries) on the export of MARC database (this is several million records). I hope that in the coming days we will settle the copyright question.

Therefore, such a script would be interesting for us.

Awesome :)

Alleycat80 updated the task description. (Show Details)Apr 1 2016, 9:19 AM

Alleycat80 updated the task description. (Show Details)Apr 1 2016, 9:21 AM

Smalyshev added a subscriber: Smalyshev.Apr 1 2016, 9:35 AM

Velodremo added a subscriber: Velodremo.Apr 1 2016, 12:18 PM

In T131417#2167440, @putnik wrote:

Wikimedia RU currently has a preliminary agreement with the Russian State Library (one of the largest Russian libraries) on the export of MARC database (this is several million records). I hope that in the coming days we will settle the copyright question.

Therefore, such a script would be interesting for us.

Great, join us at Caesarea room!

WMDE-leszek added a subscriber: WMDE-leszek.Apr 12 2016, 1:50 PM

Edgars2007 added a subscriber: Edgars2007.Apr 17 2016, 7:59 AM

Quiddity mentioned this in T135422: Wishlist tasks suggested for the Wikimania Hackathon 2016.May 31 2016, 6:49 AM

Quiddity added a project: Wikimania-Hackathon-2016.Jun 9 2016, 5:32 AM

Quiddity moved this task from Backlog to Projects on the Wikimania-Hackathon-2016 board.Jun 9 2016, 5:42 AM

johl removed a subscriber: johl.Mar 22 2017, 12:13 PM

Dvorapa added a project: Pywikibot-Wikidata.May 29 2018, 5:13 PM

Alchimista added a subscriber: Alchimista.Aug 13 2018, 4:06 PM

edwardbetts added a subscriber: edwardbetts.Nov 4 2018, 10:52 AM

• Phabricator_maintenance added a project: Pywikibot.Mar 23 2019, 10:15 PM