User Story: “As a developer, I want to know the complete list of Wikitext templates for 6 languages (EN, FR, DE, ES, IT, PT)
so that I can build a flexible algorithm to identify all citations in any language and then build a Reference parser.”

TODO

• Read PRD for references, in particlar the part on Templates.
• Sync with Stephanie and Ricardo.
• Gather a list of templates/attributes from Editing and Other sources.
• Document

Acceptance criteria
* Generate a list of most used templates of templates and it's key attributes
* Should cover 6 SC languages, EN, DE, FR, ES, IT, PT

Context

• Create a CSV file that will be our unified list of citation templates we will include in our citation parsing. The CSV will have two columns: Template Name, Template Type type is optional
• Create a file with the template names form Giovanni's Python project
• Add templates names from Go code
• Write code for a "Wikitext citation finder", we need to find all the <ref> tags in Wikitext articles. And extract the first two parts of the template: <ref>{{TemplateName TemplateType .....}}</ref> saved as a CSV TemplateName, TemplateType \n. Only save unique template names/types, ignore duplicates. Note, some <ref> tags have no wikitext template, just plain text, ignore these plain text references.
• Extend the "Wikitext citation finder" to discover citations that aren't enclosed inside the <ref></ref> tags. Look at this article https://en.wikipedia.org/w/index.php?title=Josephine_Baker&action=edit, scroll to the bottom and find sections that have {{cite... }} but don't have <ref></ref>. This is an article-level citation, that is not rendered as part of a reflist template. This one is easy to find because {{cite }} is in English, but the template name could be translated or renamed by Lua developers. These article-level citations can be inside a {{refbegin ...}} ... {{refend}} template, in a Wikitext list, or in a section heading see next ToDo item
• Optionally, extend the finder to look for short repeated templates (without much-surrounding text) in sections at the bottom of the page, that could be in "References/External Links/... (see this Go code that has found reference headers in some languages. The aim with this last ToDo is to think outside the box and look for edge cases where we could pick up more citations, leaving the details vague so developer can get creative

Test Strategy

N/A

Description

We need a definitive list of templates to find. There are a lot of projects with undocumented citation template names
Many smaller projects have translated citation templates into their language, so adding new languages will mean we need a Wikitext citation parser that can discover undocumented citations.
Once we have that list and template finder code, we can easily upgrade the cite parser to use new languages.

Citation Primer

There are a few kinds of citations in Wikitext:

1. Most newer citations are wrapped inside a <ref>{{templateName TemplateType ...}}</ref> reference markup. These are normally inline in a sentence or paragraph. The {{reflist}} template is used to render these references at the bottom of the article, the inline sentence gets rendered with a [1234] number which links to the bottom of the page reference.

2. Some editors are lazy and use the ref tag with plain text, not a template inside. We have a Go parser to approximate the structured data for this plain text.

3. Older articles didn't use the <ref> tag, they used the {{cite ...}} template (and its translations) at the bottom of the page.

{F57585564}

4. Languages have their template names for citations, this is the Turkish page for Josephine Baker, see the citation template names/types. We'll need to extract these language specific templates

Things to consider:

• Maybe split the Wikitext parser into a second Phab ticket?