We use the node.js html-metadata library to scrape metadata from webpages. This metadata is then used to generate a citation for the webpage using citoid. Editors can use this citation when they're writing or editing an article on Wikipedia as a reference.

For this you will need to install node and npm: https://www.joyent.com/blog/installing-node-and-npm/.

You will also need to fork the html-metadata library on github: https://help.github.com/articles/fork-a-repo/

application/ld+json tags are a type of metadata in a webpage that gives information about the resource. There is a good explanation about them here: https://developers.google.com/schemas/formats/json-ld

Here's an example of a page with json-ld metadata: view-source:https://www.uber.com/en-GB/

Here's a sample tag from that page:

<script type="application/ld+json">{
  "@context": "http://schema.org",
  "@type": "Organization",
  "url": "https://www.uber.com",
  "logo": "https://d1a3f4spazzrp4.cloudfront.net/uber-com/1.2.15/d1a3f4spazzrp4.cloudfront.net/images/uber-serp-logo-f6e7549c89.jpg"
}

It should also be able to handle more complex structures, such as the one found in : http://allrecipes.com/recipes/80/main-dish/

You should create a function called exports.ldJson in https://github.com/wikimedia/html-metadata/blob/master/lib/index.js that scrapes imports the contents of the json+ld tag, and creates a javascript object. You should also register the method in https://github.com/wikimedia/html-metadata/blob/master/index.js

You should also create tests for your new function. You can put tests of a live website in https://github.com/wikimedia/html-metadata/blob/master/test/scraping.js. You can also create tests of a static website in https://github.com/wikimedia/html-metadata/blob/master/test/static.js. You may find it helpful to write these tests early on in writing your function, as they will help you verify your function is working correctly. Tests can be run by doing

npm test

The original feature request was filed here: https://github.com/wikimedia/html-metadata/issues/39, so you can mention that issue in your pull request as well as the phabricator task number.