Overview

Sources are a fundamental component of Wikipedia with their reliability underlying each of the core content policies (No Original Research; Neutral Point-of-View; Verifiability). They are only loosely structured and standardized on Wikipedia, however, which makes analyzing them and building tooling around them quite difficult. The norm (not always followed) for a source is a <ref> tag within which a {{cite}} template (of which there are many) is used to wrap the various fields that define a source -- i.e. authors, date, title, publisher, URL, etc.

In reality, if you wanted to e.g., analyze the prevalence of breaking news edits on Wikipedia using the date field of sources (and comparing that to edit timestamps), you could do this fairly easily for a few language editions by going through their common citation templates and determining what format the dates take. But extending this to other languages would require further in-depth research into parameter names and norms. The best source of what these template names and parameters are comes from Citoid -- the extension that helps editors to automatically extract reference details from webpages and compile them into appropriate citation templates. These details are spread across numerous configuration pages and template code, which means that a process for consistently extracting them is needed. Furthermore, it's unclear how comprehensive the data is as far as covering references in each language of Wikipedia.

Task

The goal of this task is to build a mapping of standard citation template names for each language of Wikipedia and mapping of common parameter names to their English equivalents. This will enable researchers to define their fields of interest for one language and be able to extend any analyses with much greater ease to other languages. An initial evaluation of the coverage of this mapping will also be important to determine its effectiveness.

An example of the steps that will be required:

• The list of standard citation templates for English Wikipedia can be found in this JSON: https://en.wikipedia.org/wiki/MediaWiki:Citoid-template-type-map.json
  • Example API call for fetching: https://en.wikipedia.org/w/api.php?action=query&prop=revisions&titles=MediaWiki:Citoid-template-type-map.json&rvslots=*&rvprop=content&format=json&redirects
  • You can use this pattern for any other language where the map exists -- e.g., ar:MediaWiki:Citoid-template-type-map.json works for Arabic Wikipedia even though it has a localized name there.
• Each of the templates listed in the Citoid template type map should then have a local TemplateData page (documentation). For example, Citoid's category of journalArticle maps to Cite journal in the Citoid template type map on English Wikipedia. This refers to en:Template:Cite_journal, whose documentation can be found at en:Template:Cite_journal/doc. Examining the source code for that page will reveal a maps['citoid'] property with a mapping of standardized parameter names from Citoid to their local cite template equivalents -- e.g., the date of the publication can be found under the date parameter for this template (but e.g., in its Spanish equiavalent, it's under fecha). The TemplateData API allows you to access this information easily -- e.g., Example API call for en:Cite journal.
  • You'll also want to gather the valid redirects to each standard template -- e.g., en:Template:Cite journal has several.
• Using the APIs, you can check each Wikipedia language edition and gather their list of core citation templates and mapping of Citoid fields to parameter names. Be careful to also collect the valid aliases for each parameter name. Altogether, this should result in two dictionaries that look something like:

# mapping of Citoid standard template types to localized template names
{
  "journalArticle": {
    "en": ["Cite journal", "Citepaper", ...],
    "es": [...], 
     ...
  }
}

# mapping of Citoid fields to their localized names in each language edition
# NOTE: probably this should be further broken down by template type (journalArticle, etc.)
# but in reality it seems highly unlikely that e.g., `fecha` would be used for `date` in one citation
# template on Spanish Wikipedia but for something else in a different citation template.
# So in practice, I think it's acceptable to instead just build the full set of parameter names used
# for each Citoid field across all citation templates for a given wiki.
{
  "date": {"en": ["date"], "es": ["fecha", "date"], ... },
  "author": ...
}

Note: some properties like author actually are not a single parameter but an ordered list of author parameters. You'll want to maintain this group/ordering in the final dictionaries.

Once this mapping is put together, the next stage is to test it out! You'll want to write Python code that takes the wikitext for a page, extracts all the known cite templates, and then extracts all the fields that it can for each citation. I recommend hosting this on PAWS so it'll be easy to iterate, share code, and you have local access to data dumps for larger-scale testing. You can use the revisions API to gather some example pages -- e.g., Example API call for 5 random articles on English Wikipedia.

Once you have the code written, you'll want to manually verify the outputs for several example articles. Ideally not just in English but in one or two other languages as well. Then once you've verified that all seems to be working, you can try scaling it up to get a sense of coverage. For that, I'd either use the full current text dumps or just extend the random API to ~500 examples per wiki (I think you should be able to get 50 examples at a time with each API call). You can use the number_of_citations output from wikitext_to_refs in this notebook to get a sense of coverage -- e.g., percentage of references for each wiki in which you can extract the date field.

This task is considered [medium]. In general, it's expected that the task will take a a month or two of consistent work and is a good fit for someone with some prior coding experience and interest in being involved in research. The actual time needed, however, will depend greatly on your level of experience.

Recommended Skills

• Python coding is recommended for collecting the data and running the analyses
• Basic descriptive statistics are likely enough when it comes to extraction coverage for each Citoid field.

Acceptance Criteria

• The output of this task will be a Meta report describing the research and findings (example).
• All code and analyses should be run on PAWS (Wikimedia-hosted Jupyter notebooks) or be in a public, open-source-licensed code repository.

Process

• If you are interested in this task and it is not assigned to anyone, you may begin work on it. Please leave a comment on the task and tag @Isaac so that he is aware.
• If you have made some progress on the task (the initial validation of methods) and would like to continue, share a link to your current draft and let @Isaac know so that he can assign the task to you and help you to plot out the next steps.
• Generally, @Isaac will be able to answer any questions about the task and try to respond quickly when clarification is necessary but response times may be slow if help is needed for more general debugging etc.

Additional Resources

• How to "add a project" to Meta (@Isaac will let you know when it's a good point to do this): https://meta.wikimedia.org/wiki/Research:Projects
• Background on sources: https://meta.wikimedia.org/wiki/Research:Analyzing_sources_on_Wikipedia