Page MenuHomePhabricator

make doc files in git more useful
Closed, DuplicatePublic

Description

The docs directory in core contains various documentation files, e.g. design.txt. doxgen turns this into garbled nonsense in our generated doc at doc.wikimedia.org ☹.

There are several ways to make a documentation file in git more useful than simply a file you view in your code editor:

  1. Fix doxygen so it doesn't treat foo.txt as C++ and garble it (possibly the same as T106116: The Doxygen version in CI parses README files as garbled C.).
  2. Prepend /* and add doxygen structural commands, as includes/filebackend/README does, so that it looks good in doxygen (but note T87796: Add includes/filebackend/README as generated documentation page).
  3. Rename it design.md and use markdown formatting (and light doxygen structural commands) so it looks good in doxygen.
  4. Rename it design.mediawiki and a person (and eventually T91626: Technology to transclude git content into wiki pages) puts it on mw.org, c.f. how wikibase/docs/lua.wiki in git is copied to https://www.mediawiki.org/wiki/Extension:Wikibase_Client/Lua .
    • doxygen doesn't support wiki syntax.
    • but github supports some mediawiki syntax when rendering a file with extension .mediawiki (sample).
  5. Krinkle on IRC: I think we should consider seeking alternatives to Doxygen.

Note: Any bikeshedding here to settle on approaches should never block improving documentation in git, such as T111283: Replace docs/design.txt with a reasonably complete summary of how MW works.

Event Timeline

Spage created this task.Sep 3 2015, 1:25 AM
Spage raised the priority of this task from to Needs Triage.
Spage updated the task description. (Show Details)
Spage added subscribers: Spage, hashar, RobLa-WMF and 2 others.
Restricted Application added a project: Documentation. · View Herald TranscriptSep 3 2015, 1:25 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

As I said on T111283, if we keep these files around in Git, I think we should consider just making them pointers to mediawiki.org. If nothing else, MediaWiki is actually fairly well-suited to being used for technical documentation, in my opinion. MediaWiki supports easy linking, syntax highlighting, text formatting, an auto-generated table of contents, categorization, etc. We should leverage that instead of resorting to Markdown files or similar solutions, I think.

As I said on T111283, if we keep these files around in Git, I think we should consider just making them pointers to mediawiki.org.

I agree with you on the advantages of using mediawiki.org to document mediawiki, but doesn't that sort of defeat the purpose of having self-contained documentation included with the download? Or are you thinking of all these links pointing to the PD Help namespace which IIRC gets included in the distribution? Even so, the pages would not be readable without first installing the wiki, which may not be optimal (but is arguably a minor concern).

We could alternatively do away with docs/ files checked into git, and just use structured code comments to build a reference which would complement the more prose-like documentation in mw.org, but I'm not sure the split can be made cleanly in every instance.

hashar added a comment.Sep 3 2015, 1:07 PM

@MZMcBride and @waldyrious having the doc in git or on mediawiki.org is out of the scope of this task which is about enhancing rendering of the doc available in the source repo. An example is the docs/hooks.txt which is updated atomically with the code changing hooks. So it makes sense to have the doc in sync with the code.

daniel added a comment.Sep 3 2015, 1:20 PM

@hashar One option is to let MediaWiki render the doc, be transcluding it via a special parser function.

waldyrious added a comment.EditedSep 3 2015, 1:29 PM

An example is the docs/hooks.txt which is updated atomically with the code changing hooks. So it makes sense to have the doc in sync with the code.

That's a concern I share, but does that actually happen in practice? We currently have no systematic way of ensuring documentation is updated along with the code it refers to (it's not even in a suggested guideline). If we restrict documentation in the git repos to the structured, inline type (which would make this task irrelevant, so it's not entirely off-topic), then automated checks can be implemented to ensure docs and code are at least superficially in sync.

If indeed we manage to ensure that free-form docs are atomically updated with code, then the argument should apply to the rest of the documentation on mw.org, which would mean we'd lose mediawiki's advantages as mentioned by @MZMcBride, and would have to (re)implement at least a subset of those in doxygen.

Either case, it seems to me, would involve incurring in work that could otherwise be avoided.

Legoktm added a subscriber: Legoktm.Sep 3 2015, 3:19 PM

An example is the docs/hooks.txt which is updated atomically with the code changing hooks. So it makes sense to have the doc in sync with the code.

That's a concern I share, but does that actually happen in practice? We currently have no systematic way of ensuring documentation is updated along with the code it refers to (it's not even in a suggested guideline). If we restrict documentation in the git repos to the structured, inline type (which would make this task irrelevant, so it's not entirely off-topic), then automated checks can be implemented to ensure docs and code are at least superficially in sync.

We have the findHooks.php script which detects if docs/hooks.txt is in sync with the code.

I think it comes down to making sure reviewers -1 if docs aren't updated in patches. We do a good job of making sure docs/hooks.txt is, and I also try to make sure docs/extension.schema.json is as well.

That's good to hear. I wonder if that approach (i.e. script-based) can be extended to all free-form documentation checked into git, and if so, whether it can be run automatically on each push for review -- like TravisCI does on Github. If that is both possible and planned, I would support not only keeping the current docs in git, but also moving mw.org code there. However, I suspect the less structured the documentation is, the harder it is to do automated checks like this, hence my inclination not to rely on review culture alone to keep docs in sync with code (especially since git-hosted docs are harder to edit for occasional contributors, when the very fact that they're free-form makes such contributions more likely).

Tgr added a subscriber: Tgr.Sep 21 2015, 11:21 PM

Rename it design.md and use markdown formatting...

Yes please use Markdown (or CommonMark to be more specific). It works everywhere - in Doxygen, in whatever doc generator we'll replace it with, in GitHub, in a decent IDE, it can even be displayed in MediaWiki without much trouble if we need that (there are a number of Markdown parsers for PHP). It's also more readable in its raw form than wikitext (<code> and <source>, ugh).

I agree with @Tgr and @Spage that using Markdown is probably a good idea. The IETF Apps Area working group is unintentionally(?) helping to solidify Markdown's status as a de facto standard for wiki markup by firming up the definition of the text/markdown media type. I have no reason to see why it won't happen, and I think it's a good thing. (My ulterior motive for looking this up is making a case for T111306: Recruit event scout for November IETF meeting in preparation for #WikiDev16)

Tgr added a comment.Sep 22 2015, 1:49 AM

See also T112999: Let MediaWiki operate entirely without wikitext which makes the case for (amongst other things) Markdown-based MediaWiki as an option for third parties. With ContentHandler it is fairly easy to add new formats to MediaWiki and Markdown support in core looks like a good idea, and it would allow T91626: Technology to transclude git content into wiki pages to work seamlessly with .md files.

Krinkle updated the task description. (Show Details)Sep 18 2019, 4:49 PM
Krinkle updated the task description. (Show Details)Sep 18 2019, 4:53 PM
apaskulin removed apaskulin as the assignee of this task.Sep 18 2019, 5:01 PM
apaskulin added a subscriber: apaskulin.

The various bugs mentioned here were fixed as part of T232104. As well as a better understanding of how to conveniently add pages to Doxygen without hacky C or PHP dummy files. I'm merging this for now.

  • The task of indexing docs/ and publishing them as readable pages on doc.wikimedia.org is tracked as part of T233244.
  • The specific issue of README files outside docs/ for filebakcned is tracked as part of T87796.
  • The task of writing new documentation about the design of MediaWiki, is tracked as T111283.
hashar removed a subscriber: hashar.Dec 10 2019, 1:05 PM