Page MenuHomePhabricator

Jenkins: Generate CSS docs from LESS and publish to doc.wikimedia.org
Closed, DeclinedPublic

Description

Core has a build system to generate HTML from kss-formatted CSS/LESS (docs/kss). It would be nice if this documentation appeared in doc.wikimedia.org along with other generated documentation.

This sounds like T50337. Presumably there's some Zuul job somewhere building the doc that appears.


Version: unspecified
Severity: enhancement

Details

Reference
bz58620

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 2:31 AM
bzimport set Reference to bz58620.
bzimport added a subscriber: Unknown Object (MLST).

S, as a first step we need a way to deploy kss on the Jenkins slave.

It seems that is a nodejs application so I guess we want to use our own repository that would provide kss and ALL the required modules. We did such a thing to provide ruby bundler (integration/bundler.git) and phpunit (integration/phpunit.git).

Could you take care of having an integration/kss.git repository created and upload there the kss + its npm modules ?

Once done, register that repository in git-deploy configuration. That is done in operations/puppet.git under manifests/role/deployment.pp. The deployment would then be done from tin.eqiad.wmnet.

Then, we need a shell script wrapper that would execute the script (deployed somewhere under /srv/deployment/integration/kss ). https://doc.wikimedia.org/ document root is under /srv/org/wikimedia/doc/ the following sub directories are usually: <project-name> / <branch> / <doc component>. Example:

/srv/org/wikimedia/doc/mediawiki-core/REL1_22/js

So I guess you would want to have kss generate the doc to /srv/org/wikimedia/doc/mediawiki-core/master/css

Once done, we need a new job-template in Jenkins Job Builder configuration, apply it to mediawiki/core project. The job will need to be added in Zuul configuration (integration/zuul-config.git) in the postmerge pipeline.

Target url should indeed me something like one of these this:

https://doc.wikimedia.org/mediawiki-core/master/less/
https://doc.wikimedia.org/mediawiki-core/master/css/
https://doc.wikimedia.org/mediawiki-core/master/style-guide/css/

Preferences, anyone?

(where master can also be REL1_22, and in the future tags like 1.23.0)

With a shortcut from the https://doc.wikimedia.org/ home page, of course.

PS: I'm planning on renaming /mediawiki-core to /mediawiki at some point (with redirects and symlinks of course).

Change 103095 had a related patch set uploaded by Spage:
Add integration/kss to deployment repo config

https://gerrit.wikimedia.org/r/103095

I have created a .gitreview file for integration/kss.git repository so we can add it in the deployment system.

Change 103095 merged by Alexandros Kosiaris:
Add integration/kss to deployment repo config

https://gerrit.wikimedia.org/r/103095

Pending upload of kss in integration/kss.git so we can get it deployed on Jenkins slaves. Then we need some script to generate the doc and move the generated doc under the doc website.

Since mss is a node module, we could have it added to the integration/jenkins.git repository which has tools/package.json. It currently provides:

{
"name": "wikimedia-integration-jenkins",
"version": "0.2.5",
"dependencies": {

		"grunt": "0.4.1",
		"grunt-cli": "0.1.13",
		"grunt-contrib-qunit": "0.4.0",
		"phantomjs": "~1.9.0-1",
		"grunt-contrib-csslint": "0.2.0",
		"jshint": "2.4.4"

}
}

So maybe add kss in there and forget about git-deploy and integration/kss.

  • Bug 64221 has been marked as a duplicate of this bug. ***

In the interim, ssh tools-login.wmflabs.org; become styleguide is generating http://tools.wmflabs.org/styleguide/ in a cron job. I don't understand misc.yaml, mediawiki.yaml, zuul-config/layout.yaml enough to work on this.

yardoc is now being used and Jenkins is populating https://doc.wikimedia.org/rubygems/mediawiki-ruby-api/ and https://doc.wikimedia.org/rubygems/mediawiki-selenium/ . So that's something to study when implementing this.

Can you reach out to Timo about it? I think for Javascript purposes he would like developers to define npm entry points. So we would simply invoke 'npm doc' or something and publish whatever is generated under a given dir (ex: /doc/).

That would let you define whatever you want to run to build documentation which is much more scalable than having to handle defines in jenkins.

Timo is in the office this week.. Maybe you could grab him S?

(In reply to Antoine "hashar" Musso from comment #11)

Can you reach out to Timo about it? I think for Javascript purposes he
would like developers to define npm entry points. So we would simply invoke
'npm doc' or something and publish whatever is generated under a given dir
(ex: /doc/).

Do you mean Grunt (which runs on node.js)? That's the node.js build tool I normally see used.

Do you mean Grunt (which runs on node.js)? That's the node.js build tool
I normally see used.

I guess npm as an entry point for jenkins (i.e. npm test) which would then use grunt to run a wide range of different tests :d

Change 161998 had a related patch set uploaded by Prtksxna:
kss: Add Gruntfile

https://gerrit.wikimedia.org/r/161998

(In reply to Antoine "hashar" Musso from comment #14)

Do you mean Grunt (which runs on node.js)? That's the node.js build tool
I normally see used.

I guess npm as an entry point for jenkins (i.e. npm test) which would then
use grunt to run a wide range of different tests :d

Hmm, thanks. This is not a test though.

While "npm test" exists (https://www.npmjs.org/doc/cli/npm-test.html), "npm doc" does not. There is apparently a way to have custom scripts (https://www.npmjs.org/doc/misc/npm-scripts.html , https://www.npmjs.org/doc/cli/npm-run-script.html), but we would need to standardize on a name (e.g. npm-run-script doc).

Is there such a convention already?

Seems fine: npm-run-script doc

In Jenkins Job Builder, I have generic templates for python tox and ruby bundler:

'{name}-tox-{toxenv}'
'{name}-bundle-{bundlecommand}' https://gerrit.wikimedia.org/r/#/c/160983/

So we can probably come up with a '{name}-npmrunscript-{npmscript}' command which would be realized as:

  • project: name: VisualEditor npmscript:
    • doc jobs:
    • '{name}-npmrunscript-{npmscript}'

And the template would contain:

npm run-script {npmscript}

Might need to add an 'npm install' before, not sure.

Prtksxna updated the task description. (Show Details)Dec 5 2014, 2:40 PM
Prtksxna set Security to None.
demon removed a subscriber: demon.Dec 5 2014, 3:20 PM
Danny_B renamed this task from Jenkins: Generate KSS docs from LESS and publish to doc.wikimedia.org to Jenkins: Generate CSS docs from LESS and publish to doc.wikimedia.org.May 23 2016, 5:44 PM
Danny_B added a project: Documentation.
Jdlrobson closed this task as Declined.May 23 2016, 5:45 PM

We are no longer using kss in MobileFrontend. It wasn't being maintained due to lack of visibility and shift to OOjs UI.

Mattflaschen-WMF reopened this task as Open.Jun 2 2016, 11:51 PM

kss is still in core.

Jdlrobson closed this task as Declined.Jul 25 2018, 2:33 AM
Jdlrobson added a subscriber: Volker_E.

Old school cc @Volker_E