Maniphest T206755

Understand steps to get structured data from Excel-workbook into Wikidata
Closed, ResolvedPublic
Actions

Assigned To

None

Authored By

	OlafJanssen
	Oct 11 2018, 11:26 AM

Description

{F26519479}I have an Excel workbook (with 5 sheets) containing structured data about Dutch literary awards

For each award:

Overall info about the award: name, description, years, initiator, sponsors, organizors, namegivers, Wikipedia-URL, Wikidata-Qnumbers etc
Detailed info per year: prize money, date and venue of award ceremony, related WP and WD URLs
Nominees (short list): author names, title of nominated work, related WP and WD URLs
Winners: author names, title of winning work, related WP and WD URLs
Jury members: names, roles, related WP and WD URLs

See this sample Excel to get a better understanding, it describes the BookSpot award (and its predecessors)

This is a much richer dataset than currently listed in most Wikidata-items about Dutch literary awards (like in this item about the BookSpot award)

I want to gain a better understanding of the steps needed to bring the data from the Excel into Wikidata

Event Timeline

OlafJanssen created this task.Oct 11 2018, 11:26 AM

OlafJanssen updated the task description. (Show Details)

Come see me at some point, I can help you!
Basically:

formating your spreadsheet in a Wikidata-friendly way
use CSV2QS to convert to QuickStatements (if the data is too complex to be handled directly by QS)
run QuickStatements

Edit: the most difficult part is usually not to learn how to format the spreadsheet, it's identifying all relevant Wikidata items.

OlafJanssen removed Harmonia_Amanda as the assignee of this task.Oct 11 2018, 11:33 AM

OlafJanssen updated the task description. (Show Details)

OlafJanssen updated the task description. (Show Details)Oct 11 2018, 11:38 AM

Thanks for the very fast response Harmonia_Amanda.

I'm not yet familiar with QS, but I'll check it out before the event.

I'll also gather a list of all relevant WD-items (both already exisiting and yet to be made) related to that Bookspot award (actually, they are already in the Excel..)

I seemed to have removed you from this task, which I did not intend. I'll assign it to you again...

This tool (https://tools.wmflabs.org/ash-dev/wdutils/csv2quickstatements.php) has examples on how to format things. The missing items can be created using the same method then used to complete the rest.

Last week I discovered OpenRefine (www.openrefine.org), which was pretty much exactly the tool I was looking for. It has built in support for Wikidata and QuickStatement, and you can very easily publish data in an Excel to Wikidata.

Hi guys,

I would very much like to join you, since I have a similar task I want to work on.
See T207839
Olaf, I have started working with OpenRefine to, but if you could show me how reconciliation work I would really appreciate it!

See you on Friday!

Hi SIryn,

reconciliation is exactly what I'm finding/fiddling out at this moment as well. I find these manuals quite useful:

https://www.projectcest.be/wiki/Publicatie:Collectiedata_normaliseren_met_Open_Refine (esp the GREL functions at the bottom)
https://www.projectcest.be/wiki/Bestand:20150430_Handout_Open_Refine_workshop.pdf (for getting person data out of VIAF)
https://libjohn.github.io/openrefine/index.html (working with that right now)

Hi!

I will be available to help with OpenRefine. It is exactly designed for this workflow indeed so I hope it will be a match :)
For reconciliation help, have you seen this page?
https://github.com/OpenRefine/OpenRefine/wiki/Reconciliation

If there is anything unclear there I would be interested to know how to improve it :)

Antonin

Great, thank you @Pintoch and @OlafJanssen! I look forward to meeting you!

Dja subscribed.Oct 27 2018, 7:12 AM

This page list useful info and docs for reconcilation https://www.projectcest.be/wiki/Publicatie:Collectiedata_normaliseren_met_Open_Refine

This is a dataset with Dutch people related to literary awards https://drive.google.com/file/d/1bknyrzNYdk2wNsm6ls9cp7o6eS4nC8Vl/view?usp=sharing

Elvalente subscribed.Oct 27 2018, 7:50 AM

Excel with winners of Dutch literary awards 1881-2015 https://drive.google.com/file/d/1-T0mrZcmhFuSVNDQ8v3xM75RHKLs9K5g/view?usp=sharing

uploaded the first 50 writers from the 6000+ on the file!

Woooohooooo!!!!

Pintoch awarded a token.Oct 27 2018, 1:09 PM

Pintoch rescinded a token.

Example of item created https://www.wikidata.org/wiki/Q57826098

Ecritures moved this task from Backlog to Sessions Wikidata (Newbee friendly) on the Wikistorm-2018 board.May 22 2019, 3:59 PM

Ecritures edited projects, added Wiki-Techstorm-2019; removed Wikistorm-2018.Jun 6 2019, 7:39 PM

Pintoch added a project: OpenRefine.Jun 16 2019, 9:09 AM

This item can be closed, I know by now how to to achieve the goals mentioned using OpenRefine indeed.

OlafJanssen closed this task as Resolved.Jun 17 2019, 10:56 AM

	F26889828: afbeelding.png
	Oct 27 2018, 2:21 PM

	F26889597: afbeelding.png
	Oct 27 2018, 2:18 PM

	F26887125: Knipsel.PNG
	Oct 27 2018, 1:48 PM

Understand steps to get structured data from Excel-workbook into WikidataClosed, ResolvedPublicActions

Description

Event Timeline

Understand steps to get structured data from Excel-workbook into Wikidata
Closed, ResolvedPublic
Actions