Page MenuHomePhabricator

Define entry point to generate documentation
Closed, ResolvedPublic

Description

We are migrating Jenkins jobs to simply invokes test entry points (ex: composer test, npm test). More and more repositories are willing to have documentation generated and published to https://doc.wikimedia.org/ . The problem is that each repository ended up with its own set of command which causes us to have a job per repository. Some examples:

Python projects tox -edocs
OOjsjsduck
OOjs UIjsduck
VisualEditorjsduck
GuidedTourjsduck
MobileFrontendmake docs
MultimediaViewer./docs/generate
mwext-VisualEditor./bin/generateDocs.sh
MediaWikicustom script around maintenance/mwdocgen.php

We could reuse the test utilities (tox, npm, composer ..) but some repositories have documentations in different languages (jsduck / sphinx / whatever) which we will want to generate together for publishing.

Maybe we could establish the convention of using a MakeFile. It is simple enough to write and can shell out to whatever underlying command people might want to use (ex: make doc).

Finally, docs are generated to arbitrary paths ( docs/, build/docs, .docs/html, /html/docs ...). The Jenkins template has support to specify the directory to copy from. Ultimately it would be nice to agree on a convention, a simple one would be to have the make doc script to recognize an env variable containing the doc destination (ex: WMF_CI_DOC_OUTPUT), which will make all Jenkins jobs the same.

Details

Related Gerrit Patches:
integration/config : masterJob templates for `make docs`

Event Timeline

hashar raised the priority of this task from to Needs Triage.
hashar updated the task description. (Show Details)
hashar added a subscriber: hashar.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 9 2015, 5:07 PM

Alternatively, we can make the command to be a variable passed to the job.

I have posted on wikitech-l https://lists.wikimedia.org/pipermail/wikitech-l/2015-February/080670.html

I will use the convention 'make docs' unless wikitech-l discussions says otherwise.

hashar triaged this task as Medium priority.Feb 9 2015, 5:19 PM
hashar set Security to None.
daniel added a subscriber: daniel.
scfc added a subscriber: scfc.Feb 9 2015, 6:38 PM
Qgil added a subscriber: Qgil.
Krinkle updated the task description. (Show Details)Feb 9 2015, 11:00 PM
greg added a subscriber: greg.Feb 10 2015, 5:52 PM
Ltrlg added a subscriber: Ltrlg.Feb 10 2015, 10:19 PM
hashar closed this task as Resolved.Feb 17 2015, 1:59 PM

After a week on wikitech-l, make docs is the documentation entry point from now on.

I have updated the wiki page.

What should the output directory be?

gerritbot added a subscriber: gerritbot.

Change 191045 had a related patch set uploaded (by Hashar):
Job templates for make docs

https://gerrit.wikimedia.org/r/191045

Patch-For-Review

What should the output directory be?

We can configure the output dir to publish in the Jenkins job. It is the docsrc JJB variable which is then passed to the job template and the doc-publish macro which does the rsync.

A work in progress example for MobileFrontend is https://gerrit.wikimedia.org/r/#/c/191046/3/jjb/mediawiki-extensions.yaml

In short:

docsrc: whatever/you/want

That being said, we can probably use a default value there (such as $WORKSPACE/docbuild ) and export the path as an env variable (ex: WMF_CI_DOC_OUTPUT_DIR ) which can be recognized by developers in their docs generation scripts.

jayvdb added a subscriber: jayvdb.Feb 19 2015, 10:49 PM

Change 191045 abandoned by Hashar:
Job templates for make docs

Reason:
Was a one off discussion

https://gerrit.wikimedia.org/r/191045