Page MenuHomePhabricator

Enable jenkins test & submit for graphoid repo
Closed, ResolvedPublic

Description

graphoid service repo should be tested by jenkins and merged on +2 if passes.

The repository is a standard node.js backend service similar to Parsoid/Mathoid/cxserver etc..

Event Timeline

Yurik created this task.Jul 23 2015, 11:40 AM
Yurik raised the priority of this task from to Medium.
Yurik updated the task description. (Show Details)
Yurik added projects: Graphoid, Jenkins.
Yurik added a subscriber: Yurik.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 23 2015, 11:40 AM
Yurik set Security to None.
hashar updated the task description. (Show Details)Jul 23 2015, 3:39 PM
hashar edited projects, added Continuous-Integration-Config; removed Jenkins.
hashar added a subscriber: mobrovac.

The problem is that Graphoid needs additional packages installed. @hashar can we apt-get install stuff as part of the process?

Should be as easy as editing the zuul/layout.yaml file in integration/config.git and add:

- name: mediawiki/services/graphoid:
  template:
    - name: npm

That will cause Zuul to trigger a job that runs npm install && npm test.

If some additional system dependencies are needed, list them out and we can have them installed on the slaves via puppet and the class contint::packages::labs.

Change 249439 had a related patch set uploaded (by Hashar):
[graphoid] experimental npm job

https://gerrit.wikimedia.org/r/249439

Change 249439 merged by jenkins-bot:
[graphoid] experimental npm job

https://gerrit.wikimedia.org/r/249439

The CI patch I made let you trigger the Jenkins npm job on Trusty slaves by commenting in Gerrit: check experimental.

I created a dummy change for graphoid: https://gerrit.wikimedia.org/r/#/c/249440/ and trigger such a build. It fails with:

+ npm test

> graphoid@0.1.6 test /mnt/jenkins-workspace/workspace/npm
> mocha



  express app
server options changed; restarting
npm ERR! Test failed.  See above for more details.

I have the same issue on my local machine.

Graphoid requires some extra pkgs to be installed on the system, cf. the dependencies.

I am npm illiterate, but what is that section in package.json:

"deploy": {
  "target": "ubuntu",  
  "dependencies": {
    "_all": ["libcairo2-dev", "libgif-dev", "libpango1.0-dev"], 
    "ubuntu": ["libjpeg62-dev"],
    "debian": ["libjpeg62-turbo-dev"]
  }
}

Shouldn't npm install complains about all those dependencies missing? :-)

We will need both the binaries and -dev packages dependencies to be shipped on the CI slave.

For production the packages are defined in modules/graphoid/manifests/init.pp.

If we could extract to a standalone class such as graphoid::packages that will let us include it on the CI slaves and make sure it is up-to-date.

Will need to find a way to ship the matching dev package. Maybe graphoid::packages could have a parameter to forge -dev packages and add them as well.

I am npm illiterate, but what is that section in package.json:

This is service-template-node / service-runner specific notation. We have a script that updates the deploy repo via Docker containers. This sections tells it which distro to spawn and which extra packages need to be installed before running npm install.

Shouldn't npm install complains about all those dependencies missing? :-)

I like your optimism :) But, no, it shouldn't as the deploy stanza is not an npm standard.

Will need to find a way to ship the matching dev package. Maybe graphoid::packages could have a parameter to forge -dev packages and add them as well.

Maybe simply doing what the builder script above does? Instead of just plainly running npm install we could run a script that parses package.json and installs the required packages beforehand?

hashar changed the task status from Open to Stalled.Nov 26 2015, 10:37 AM

I have created a sub task to get the packages to be provisioned via puppet T119693: Extract Graphoid Debian packages dependencies to a standalone puppet class so they can be installed on CI slaves.

Meanwhile stalling this task and marking it as being blocked.

Change 269147 had a related patch set uploaded (by Hashar):
graphoid: add noop jobs

https://gerrit.wikimedia.org/r/269147

Change 269147 merged by jenkins-bot:
graphoid: add noop jobs

https://gerrit.wikimedia.org/r/269147

Change 276445 had a related patch set uploaded (by Hashar):
graphoid: add npm-node-4.3

https://gerrit.wikimedia.org/r/276445

https://gerrit.wikimedia.org/r/#/c/274687/ switched Graphoid to Node 4.3

The Graphoid packages are now available on the CI instances (T119693)

I tested the npm job on Graphoid patch https://gerrit.wikimedia.org/r/274687 and it works.

Thus I have deployed the CI change https://gerrit.wikimedia.org/r/#/c/276445/ to enable npm :}

Still have to enable tests on the /deploy repo.

hashar changed the task status from Stalled to Open.Mar 10 2016, 12:47 PM

Change 276445 merged by jenkins-bot:
graphoid: add npm-node-4.3

https://gerrit.wikimedia.org/r/276445

Change 276447 had a related patch set uploaded (by Hashar):
graphoid/deploy experimental graphoid-deploy-npm-node-4.3

https://gerrit.wikimedia.org/r/276447

Change 276447 merged by jenkins-bot:
graphoid/deploy experimental graphoid-deploy-npm-node-4.3

https://gerrit.wikimedia.org/r/276447

Change 276449 had a related patch set uploaded (by Hashar):
Enable graphoid-deploy-npm-node-4.3

https://gerrit.wikimedia.org/r/276449

hashar closed this task as Resolved.Mar 10 2016, 1:08 PM
hashar claimed this task.

We now trigger nodeJs 4.3 based Jenkins jobs on both the source and deploy repositories. Just like the other mediawiki/services/* repositories.

Huge thanks to @mobrovac to have implemented the .deb packages provisionning in puppet.

Change 276449 merged by jenkins-bot:
Enable graphoid-deploy-npm-node-4.3

https://gerrit.wikimedia.org/r/276449