Page MenuHomePhabricator

Be able to generate citation list using citoid
Open, LowPublic

Description

Background
Citoid uses Zotero for automatic metadata generation. Zotero supports generating list of citations (e.g detectWeb function returns 'multiple' in the translator).

Examples:

Motivation
Many Wikipedia articles, especially about book authors, contains a long list of works written by the author (examples: Douglas_Adams, Isaac_Bashevis_Singer)

It would be awesome if citoid could take advantage of the already existing Zotero capability to generate list of works.

Suggestion

  1. Expose this Zotero's capability with Citoid API
  2. UI (assuming VE or new source editor): use the existing "Citation" and based on the 'multiple' result generate list of citations but outside ref (as bullet list). [IMO: this is nice as it avoids creating too many menu buttons...]

Workaround
"FYI there is a way to do this using scholia/wikidata if the author is in wikidata that generates wikitext. It does not use citoid.

https://tools.wmflabs.org/scholia/q-to-bibliography-templates

If the author is not yet in wikidata or their works or missing, you can create it using sourcemd: https://tools.wmflabs.org/sourcemd/ "

Event Timeline

Wouldn't it be better to add those lists to Wikidata, and generate lists from WIkidata? It would save time and efforts to create lists for all wikis and update them: if a new book is written, I would prefer, as a volunteer, to update only one place, not all wikis.

• Deskana lowered the priority of this task from Medium to Low.Aug 22 2017, 7:17 PM
• Deskana subscribed.

Wouldn't it be better to add those lists to Wikidata, and generate lists from WIkidata? It would save time and efforts to create lists for all wikis and update them: if a new book is written, I would prefer, as a volunteer, to update only one place, not all wikis.

Probably, yes. :-)

Wouldn't it be better to add those lists to Wikidata, and generate lists from WIkidata? It would save time and efforts to create lists for all wikis and update them: if a new book is written, I would prefer, as a volunteer, to update only one place, not all wikis.

this isn't feasible with wikidata - books have many versions and translations, each language would prefer to list either the original works or translations to that language and exclude translations to other language and it isn't possible to do that with the current infrastructure of Wikidata.
To get into more details with example, suppose that we would like to do it for Douglas Adams which is example entity in Wikidata and is quite rich with claims (https://www.wikidata.org/wiki/Q42 and the 42 is assigned intentionally to this entity). To make a list of books one can:

  1. use "{{#statements:P800}}" - not enough, as it isn't formatted as list and miss the metadata
  2. write a lua module getEntity('Q42'):getBestStatements('P800'), iterate on the books and for each of them access it's entity, fetch the metadata and format it nicely - this is almost good (except some metadata such as ISBN/publisher etc not exist on those entities). but still not good enough as one would like to mention only the original works and the translation to the relevant language - and here comes the tricky part - if each edition/translation of the books have it's own entity it requires very costly access to wikidata (Author => Book => Edition), it isn't clear how to filter the relevant editions.

I was suggesting this, because I've explored the Wikidata option with some friends (on my volunteer capacity).
Every unique book has an ISBN and it is possible to specify the language for each. Then, you just have to filter by languages and export a list.

But I acknowledge this is not the same thing than just transcluding a Wikidata query on a wiki, not feasible as we speak.

This comment was removed by Trizek-WMF.

See WikiCite for a huge amount of thinking done on this already. It would be a shame to duplicate that work.

Before we get ahead of ourselves with wikidata and what not, there are some slight complications:

We currently have a fuzzy search that allows multiple results from a single identifier. We'd have to figure out under what circumstances we allow the fuzzy multiple results versus the "true" multiple results from Zotero, or if we'd just not bother distinguishing between the two.

Secondly, "multiple" doesn't return complete citations in translation-server which is why it is not currently enabled. You tend to get a object back of just the identifiers or relative urls. For example, this is the result in translation-server for requesting a google scholar page:

curl -d '{"url":"https://scholar.google.co.uk/scholar?hl=en&q=em+volz","sessionid":"abc123"}' --header "Content-Type: application/json" 127.0.0.1:1969/web
{  
   "/scholar?q=info:OGB8lqiQ3nEJ:scholar.google.com/&output=cite&scirp=0&hl=en":"Phylodynamics of infectious disease epidemics",
   "/scholar?q=info:VTKkieKMEIMJ:scholar.google.com/&output=cite&scirp=1&hl=en":"[HTML][HTML] Effects of heterogeneous and clustered contact patterns on infectious disease dynamics",
   "/scholar?q=info:gjvYzhIOMi4J:scholar.google.com/&output=cite&scirp=2&hl=en":"Edge-based compartmental modelling for infectious disease spread",
   "/scholar?q=info:Ptt1m04FFkkJ:scholar.google.com/&output=cite&scirp=3&hl=en":"[HTML][HTML] Viral phylodynamics",
   "/scholar?q=info:kx47sfRwSbIJ:scholar.google.com/&output=cite&scirp=4&hl=en":"[HTML][HTML] Viral phylodynamics and the search for an 'effective number of infections'",
   "/scholar?q=info:JULIHyvQugcJ:scholar.google.com/&output=cite&scirp=5&hl=en":"Complex population dynamics and the coalescent under neutrality",
   "/scholar?q=info:AKp2-DhoJBQJ:scholar.google.com/&output=cite&scirp=6&hl=en":"[HTML][HTML] HIV-1 transmission during early infection in men who have sex with men: a phylodynamic analysis",
   "/scholar?q=info:Hk9cZ9dFQRgJ:scholar.google.com/&output=cite&scirp=7&hl=en":"Evolution of the coronary care unit: clinical characteristics and temporal trends in healthcare delivery and outcomes",
   "/scholar?q=info:zkEyyZbdKNoJ:scholar.google.com/&output=cite&scirp=8&hl=en":"[HTML][HTML] Using respondent driven sampling in a hidden population at risk of HIV infection: Who do HIV-positive recruiters recruit?",
   "/scholar?q=info:4Q8FsSsLHzYJ:scholar.google.com/&output=cite&scirp=9&hl=en":"[HTML][HTML] Simple epidemiological dynamics explain phylogenetic clustering of HIV from patients with recent infection"
}

It's certainly possible to do, but would require a little extra work, possibly on translation-server itself, which hasn't even been updated in production two years :/. It actually might be more expedient to actually do the expansion into citations in citoid instead.

I agree this is definitely a nice to have though, I've wanted some version of it myself. For instance, I recently used a hodgepodge of things to upload all of Maryam's works into wikidata and it was definitely not a fun process.

I should probably note that this is probably done this way in translation server because retrieving each citation individually increases request time quite a lot. We can do it in parallel but it might be too lengthy; request time is already pretty long. We also have a results limit in place currently set to 4, I think, in production? We'd need to lengthen this for this to be useful.

We could also present these multiple results only partially and use the "insert" button to get each full citation results individually, which is what the Zotero UI does. But that definitely complicates our API, we'd have to have a separate endpoint maybe for these partial multiple results.

FYI there is a way to do this using scholia/wikidata if the author is in wikidata that generates wikitext. It does not use citoid.

https://tools.wmflabs.org/scholia/q-to-bibliography-templates

If the author is not yet in wikidata or their works or missing, you can create it using sourcemd: https://tools.wmflabs.org/sourcemd/