We have content in git repositories that we want to show in wiki pages:
Documentation in git
- core/docs files
- 23 text files
- many are actually wikitext or Markdown-ish despite no extension or .txt extension (T116690: Give text files in Git correct extensions)
- hooks.txt is a special case, T93043 and now T115388: Convert hooks.txt to YAML [Outreachy microtask]
- some HTML bits in docs/html
- 23 text files
- also proposal to keep architectural module docs in git and publish to wiki
- extension doc files
- mostly README.md files (sample) whose information is duplicated on the extension's page on mw.org.
- others, e.g. ContentTranslation/hooks.md
- plus some .txt files , mostly hooks.txt
- extensions/Wikibase/docs has five big .wiki files (e.g. lua.wiki is manually copied to WikiExtension:Wikibase Client/Lua) plus summaries.txt
- The three opening paragraphs of https://doc.wikimedia.org/oojs-ui/master/js/#!/api/OO.ui.Toolbar (generated from Toolbar.js) and https://www.mediawiki.org/wiki/OOjs_UI/Toolbars were identical and are now drifting apart.
- Because github renders a subset of wiki markup when viewing path/file.wiki or file.mediawiki, developers are creating CREDITS.mediawiki, README.mediawiki, etc.
- The Wikipedia Android app's README.mediawiki has only a few sentences then "All documentation is kept on our wiki. Check it out!" This could be inverted so the README is maintained in git and this file is transcluded on the mw.org page.
Sample code from git
- extensions/examples source code
- production code that demonstrates something useful
- e.g. PHP Api request has code from Extension:WikiLove
- e.g. HTMLForm/tutorial2 has code from ??
Minimum viable product
- Develop an extension such that existing wiki page can transclude latest content from git using some syntax, probably like Git2Page's {{snippet}} parser function, with similar functionality.
- Using this editors can
- transclude wikitext into the page
- It can provide source code to the <source> tag or parser function to transclude highlighted source code into the page.
- The extension is deployed on the production wiki for mediawiki.org:
- caching strategy
- continuous integration
- security review
- etc.
Possible/desirable features
After the MVP!
- Transcluded content is available to Lua modules to operate on, e.g. transform hooks.txt into something useful.
- It should be possible to invoke this functionality from a wikitext template, so that e.g. a wiki editor can write a Template:Show_extension_README or `Template:PHP_code_snippet
- Render Markdown files (e.g. using PHP Markdown)
- Render .txt files with some minimal formatting (like the <poem> tag?)
- Render other formats if agreed (e.g. pandoc).
For sample code, editors need to transclude parts of Git files into a wiki page (see example). Possible approaches:
- In the parser tag, specify a range of lines from the file, like Git2pages' {{snippet}} parser function:
- {{#snippet:repository=mediawiki/extensions/examples | filename=examples/ContentAction/ContentAction.php |startline=24 |endline=36}}
- Could also/instead identify sections of code with starting and ending markers. Developers would annotate files in git with something like // begin name=simple-text-field and a matching // end name=simple-text-field , and then in the parser tag wiki editors specify {{#snippet: ... |section=simple-text-field` }}.
- Obviously developers might mistype, delete, or duplicate these section marker names, but it's developer error; the extension can set a hidden category "Pages with garbled Git section transclusion", or warn, or try to do the right thing.
- Could leave it up to a Lua module to get sections of the file using either technique.
Unlikely/non features
- Create sets of wiki pages from git files (e.g. doc/hooks.txt)
- if you want that, probably publish HTML to http://doc.wikimedia.org; or develop a Lua module that invokes this task's functionality and grabs the right part of the file.
- Push git content to wiki page using some kind of src->destWikiPage mapping file
- instead just have a wiki bot insert the parser function into pages
- Explore alternative syntax?: develop a special page e.g. Special:FromGit, and transclude that in other pages, e.g. {{[[Special:FromGit/project/git/path/path/to/file]]}} (This is a similar approach to {{Special:ApiHelp/query+categoryinfo}} which transcludes action API documentation.)
- it's hard to specify all the parameters. What's the actual use case for this syntax?
Prior art
- Extension:Git2Pages - execs git clone and other commands to create local files
- Extension:GitHub - issues API requests to https://raw.githubusercontent.com to get files. Pulls in 12,000 lines of composer libraries.
- https://www.mediawiki.org/wiki/Special:ApiHelp/query+categoryinfo (transcludable)
- Extension:Include - primitive single-file PHP extension simply issues file_get_contents() of a URI.
- Extension:DocBrowser from documentation overhaul RFC??
Information for Possible-Tech-Projects
- Primary mentor:
- Co-mentor: @Tgr
- Other mentors:
(optional, Phabricator username)
- Skills: PHP, Git; familiarity with MediaWiki extensions helps but can be picked up on the go
- Estimated project time for a senior contributor: (must be 2-3 weeks) OK it's 2-3 weeks :-)
- Microtasks: T114719: Convert Git2Pages to use extension registration, T115388: Convert hooks.txt to YAML [Outreachy microtask] (pick one or request another)