Enable librsvg-dev on jenkins
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Physikerwelt
	Feb 27 2016, 3:46 PM

Description

Tests fail for mathoid-mathjax-node@0.6.0 cf. https://gerrit.wikimedia.org/r/#/c/270547/

15:42:57 > librsvg@0.6.1 install /home/jenkins/workspace/npm-node-4.3/node_modules/mathoid-mathjax-node/node_modules/librsvg
15:42:57 > node-gyp rebuild
15:42:57 
15:42:57 (node) child_process: options.customFds option is deprecated. Use options.stdio instead.
15:42:57 Package librsvg-2.0 was not found in the pkg-config search path.
15:42:57 Perhaps you should add the directory containing `librsvg-2.0.pc'
15:42:57 to the PKG_CONFIG_PATH environment variable
15:42:57 No package 'librsvg-2.0' found
15:42:57 Package cairo-png was not found in the pkg-config search path.
15:42:57 Perhaps you should add the directory containing `cairo-png.pc'
15:42:57 to the PKG_CONFIG_PATH environment variable
15:42:57 No package 'cairo-png' found
15:42:57 Package cairo-pdf was not found in the pkg-config search path.
15:42:57 Perhaps you should add the directory containing `cairo-pdf.pc'
15:42:57 to the PKG_CONFIG_PATH environment variable
15:42:57 No package 'cairo-pdf' found
15:42:57 Package cairo-svg was not found in the pkg-config search path.
15:42:57 Perhaps you should add the directory containing `cairo-svg.pc'
15:42:57 to the PKG_CONFIG_PATH environment variable
15:42:57 No package 'cairo-svg' found
15:42:57 gyp: Call to 'pkg-config --libs-only-l librsvg-2.0 cairo-png cairo-pdf cairo-svg' returned exit status 1 while in binding.gyp.
15:42:57 gyp ERR! configure error 
15:42:57 gyp ERR! stack Error: `gyp` failed with exit code: 1
15:42:57 gyp ERR! stack     at ChildProcess.onCpExit (/usr/share/node-gyp/lib/configure.js:344:16)
15:42:57 gyp ERR! stack     at emitTwo (events.js:87:13)
15:42:57 gyp ERR! stack     at ChildProcess.emit (events.js:172:7)
15:42:57 gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)
15:42:57 gyp ERR! System Linux 3.16.0-4-amd64
15:42:57 gyp ERR! command "/usr/bin/nodejs" "/usr/bin/node-gyp" "rebuild"
15:42:57 gyp ERR! cwd /home/jenkins/workspace/npm-node-4.3/node_modules/mathoid-mathjax-node/node_modules/librsvg
15:42:57 gyp ERR! node -v v4.3.0
15:42:57 gyp ERR! node-gyp -v v0.12.2
15:42:57 gyp ERR! not ok 
15:42:57 npm WARN This failure might be due to the use of legacy binary "node"
15:42:57 npm WARN For further explanations, please read
15:42:57 /usr/share/doc/nodejs/README.Debian

Details

	Subject	Repo	Branch	Lines +/-
	dib: include graphoid/mathoid package dependencies	integration/config	master	+5 -0
	Services: introduce service::packages	operations/puppet	production	+85 -17

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Duplicate	None	T129107 Change png format for mathematical formulae to be more accessible
Resolved	• mobrovac	T128280 Enable librsvg-dev on jenkins
Resolved	hashar	T129092 Hiera is not properly configured on Nodepool instances

Event Timeline

Physikerwelt created this task.Feb 27 2016, 3:46 PM

Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptFeb 27 2016, 3:46 PM

Physikerwelt updated the task description. (Show Details)Feb 27 2016, 3:47 PM

Graphoid needs a few -dev packages to be provisioned as well ( T119693 ). From that task,the easiest (CI wise) is to have a puppet classe to ship packages required at runtime (ex: libcairo) and optionally the dev packages (ex: libcairo-dev). For example a class mathoid::packages which would take an option such as dev_packages => false which we can then set to True for CI instances.

The idea is to have the list of dependencies listed somewhere out of CI (i.e. in puppet.git) and at a place where people will remember to mention the needed -dev packages.

But maybe there is better way that would match both Mathoid and Graphoid use cases.

hashar mentioned this in T119693: Extract Graphoid Debian packages dependencies to a standalone puppet class so they can be installed on CI slaves.Feb 29 2016, 2:45 PM

on my development instance (running ubuntu 14 LTS) everything I did was running

sudo apt-get install librsvg2-dev

(http://packages.ubuntu.com/de/trusty/librsvg2-dev)
What would be the data that I as a Jenkins user have to provide in that case.

Physikerwelt added a subscriber: Yurik.Feb 29 2016, 4:29 PM

Graphoid dependencies tree, just in case it is needed.

yurik@steppenwolf:~/wmf/graphoid/graphoid$ npm ls
graphoid@0.1.7 /home/yurik/wmf/graphoid/graphoid
├── bluebird@3.3.1
├─┬ body-parser@1.15.0
│ ├── bytes@2.2.0
│ ├── content-type@1.0.1
│ ├─┬ debug@2.2.0
│ │ └── ms@0.7.1
│ ├── depd@1.1.0
│ ├─┬ http-errors@1.4.0
│ │ ├── inherits@2.0.1
│ │ └── statuses@1.2.1
│ ├── iconv-lite@0.4.13
│ ├─┬ on-finished@2.3.0
│ │ └── ee-first@1.1.1
│ ├── qs@6.1.0
│ ├─┬ raw-body@2.1.5
│ │ └── unpipe@1.0.0
│ └─┬ type-is@1.6.11
│   ├── media-typer@0.3.0
│   └── mime-types@2.1.9
├─┬ bunyan@1.6.0
│ ├─┬ dtrace-provider@0.6.0
│ │ └── nan@2.2.0
│ ├── moment@2.11.2
│ ├─┬ mv@2.1.1
│ │ ├── ncp@2.0.0
│ │ └─┬ rimraf@2.4.5
│ │   └── glob@6.0.4
│ └── safe-json-stringify@1.0.3
├─┬ cassandra-uuid@0.0.2
│ └── long@2.4.0
├─┬ compression@1.6.1
│ ├─┬ accepts@1.3.1
│ │ └── negotiator@0.6.0
│ ├─┬ compressible@2.0.7
│ │ └── mime-db@1.21.0
│ ├── on-headers@1.0.1
│ └── vary@1.1.0
├── domino@1.0.23
├─┬ express@4.13.4
│ ├─┬ accepts@1.2.13
│ │ └── negotiator@0.5.3
│ ├── array-flatten@1.1.1
│ ├── content-disposition@0.5.1
│ ├── cookie@0.1.5
│ ├── cookie-signature@1.0.6
│ ├── escape-html@1.0.3
│ ├── etag@1.7.0
│ ├── finalhandler@0.4.1
│ ├── fresh@0.3.0
│ ├── merge-descriptors@1.0.1
│ ├── methods@1.1.2
│ ├── parseurl@1.3.1
│ ├── path-to-regexp@0.1.7
│ ├─┬ proxy-addr@1.0.10
│ │ ├── forwarded@0.1.0
│ │ └── ipaddr.js@1.0.5
│ ├── qs@4.0.0
│ ├── range-parser@1.0.3
│ ├─┬ send@0.13.1
│ │ ├── destroy@1.0.4
│ │ ├── http-errors@1.3.1
│ │ └── mime@1.3.4
│ ├── serve-static@1.10.2
│ ├── utils-merge@1.0.0
│ └── vary@1.0.1
├── extend@3.0.0
├── graph-shared@0.0.2 (git+ssh://git@github.com/nyurik/graph-shared.git#852742aba9206e451e42383361978cbccaaeaad2)
├─┬ istanbul@0.4.2
│ ├── abbrev@1.0.7
│ ├── async@1.5.2
│ ├─┬ escodegen@1.7.1
│ │ ├── esprima@1.2.5
│ │ ├── estraverse@1.9.3
│ │ ├── esutils@2.0.2
│ │ ├─┬ optionator@0.5.0
│ │ │ ├── deep-is@0.1.3
│ │ │ ├── fast-levenshtein@1.0.7
│ │ │ ├── levn@0.2.5
│ │ │ ├── prelude-ls@1.1.2
│ │ │ └── type-check@0.3.2
│ │ └── source-map@0.2.0
│ ├── esprima@2.7.2
│ ├─┬ fileset@0.2.1
│ │ ├─┬ glob@5.0.15
│ │ │ ├── inflight@1.0.4
│ │ │ └── path-is-absolute@1.0.0
│ │ └── minimatch@2.0.10
│ ├─┬ handlebars@4.0.5
│ │ ├─┬ optimist@0.6.1
│ │ │ └── wordwrap@0.0.3
│ │ ├─┬ source-map@0.4.4
│ │ │ └── amdefine@1.0.0
│ │ └─┬ uglify-js@2.6.1
│ │   ├── async@0.2.10
│ │   ├── source-map@0.5.3
│ │   ├── uglify-to-browserify@1.0.2
│ │   └─┬ yargs@3.10.0
│ │     ├── camelcase@1.2.1
│ │     ├─┬ cliui@2.1.0
│ │     │ ├─┬ center-align@0.1.3
│ │     │ │ ├─┬ align-text@0.1.4
│ │     │ │ │ ├─┬ kind-of@3.0.2
│ │     │ │ │ │ └── is-buffer@1.1.2
│ │     │ │ │ ├── longest@1.0.1
│ │     │ │ │ └── repeat-string@1.5.2
│ │     │ │ └── lazy-cache@1.0.3
│ │     │ ├── right-align@0.1.3
│ │     │ └── wordwrap@0.0.2
│ │     └── window-size@0.1.0
│ ├── nopt@3.0.6
│ ├─┬ once@1.3.3
│ │ └── wrappy@1.0.1
│ ├── resolve@1.1.7
│ ├─┬ supports-color@3.1.2
│ │ └── has-flag@1.0.0
│ ├─┬ which@1.2.4
│ │ ├─┬ is-absolute@0.1.7
│ │ │ └── is-relative@0.1.3
│ │ └── isexe@1.1.2
│ └── wordwrap@1.0.0
├─┬ js-yaml@3.5.3
│ └─┬ argparse@1.0.6
│   └── sprintf-js@1.0.3
├─┬ mkdirp@0.5.1
│ └── minimist@0.0.8
├─┬ mocha@2.4.5
│ ├── commander@2.3.0
│ ├── diff@1.4.0
│ ├── escape-string-regexp@1.0.2
│ ├─┬ glob@3.2.3
│ │ ├── graceful-fs@2.0.3
│ │ └─┬ minimatch@0.2.14
│ │   ├── lru-cache@2.7.3
│ │   └── sigmund@1.0.1
│ ├── growl@1.8.1
│ ├─┬ jade@0.26.3
│ │ ├── commander@0.6.1
│ │ └── mkdirp@0.3.0
│ └── supports-color@1.2.0
├─┬ mocha-jshint@2.3.1
│ ├─┬ jshint@2.9.1
│ │ ├─┬ cli@0.6.6
│ │ │ └─┬ glob@3.2.11
│ │ │   └── minimatch@0.3.0
│ │ ├─┬ console-browserify@1.1.0
│ │ │ └── date-now@0.1.4
│ │ ├── exit@0.1.2
│ │ ├─┬ htmlparser2@3.8.3
│ │ │ ├── domelementtype@1.3.0
│ │ │ ├── domhandler@2.3.0
│ │ │ ├─┬ domutils@1.5.1
│ │ │ │ └─┬ dom-serializer@0.1.0
│ │ │ │   ├── domelementtype@1.1.3
│ │ │ │   └── entities@1.1.1
│ │ │ ├── entities@1.0.0
│ │ │ └─┬ readable-stream@1.1.13
│ │ │   ├── core-util-is@1.0.2
│ │ │   ├── isarray@0.0.1
│ │ │   └── string_decoder@0.10.31
│ │ ├── lodash@3.7.0
│ │ ├── minimatch@2.0.10
│ │ ├── shelljs@0.3.0
│ │ └── strip-json-comments@1.0.4
│ ├─┬ minimatch@3.0.0
│ │ └─┬ brace-expansion@1.1.3
│ │   ├── balanced-match@0.3.0
│ │   └── concat-map@0.0.1
│ ├── shelljs@0.4.0
│ └── uniq@1.0.1
├── mocha-lcov-reporter@1.0.0
├─┬ preq@0.4.8
│ └─┬ request@2.69.0
│   ├── aws-sign2@0.6.0
│   ├── aws4@1.2.1
│   ├─┬ bl@1.0.3
│   │ └─┬ readable-stream@2.0.5
│   │   ├── process-nextick-args@1.0.6
│   │   └── util-deprecate@1.0.2
│   ├── caseless@0.11.0
│   ├─┬ combined-stream@1.0.5
│   │ └── delayed-stream@1.0.0
│   ├── forever-agent@0.6.1
│   ├── form-data@1.0.0-rc3
│   ├─┬ har-validator@2.0.6
│   │ ├─┬ chalk@1.1.1
│   │ │ ├── ansi-styles@2.1.0
│   │ │ ├── has-ansi@2.0.0
│   │ │ └── supports-color@2.0.0
│   │ ├─┬ commander@2.9.0
│   │ │ └── graceful-readlink@1.0.1
│   │ ├─┬ is-my-json-valid@2.12.4
│   │ │ ├── generate-function@2.0.0
│   │ │ ├─┬ generate-object-property@1.2.0
│   │ │ │ └── is-property@1.0.2
│   │ │ ├── jsonpointer@2.0.0
│   │ │ └── xtend@4.0.1
│   │ └─┬ pinkie-promise@2.0.0
│   │   └── pinkie@2.0.4
│   ├─┬ hawk@3.1.3
│   │ ├── boom@2.10.1
│   │ ├── cryptiles@2.0.5
│   │ ├── hoek@2.16.3
│   │ └── sntp@1.0.9
│   ├─┬ http-signature@1.1.1
│   │ ├── assert-plus@0.2.0
│   │ ├─┬ jsprim@1.2.2
│   │ │ ├── extsprintf@1.0.2
│   │ │ ├── json-schema@0.2.2
│   │ │ └── verror@1.3.6
│   │ └─┬ sshpk@1.7.4
│   │   ├── asn1@0.2.3
│   │   ├── dashdash@1.12.2
│   │   ├── ecc-jsbn@0.1.1
│   │   ├── jodid25519@1.0.2
│   │   ├── jsbn@0.1.0
│   │   └── tweetnacl@0.13.3
│   ├── is-typedarray@1.0.0
│   ├── isstream@0.1.2
│   ├── json-stringify-safe@5.0.1
│   ├── node-uuid@1.4.7
│   ├── oauth-sign@0.8.1
│   ├── qs@6.0.2
│   ├── stringstream@0.0.5
│   ├── tough-cookie@2.2.1
│   └── tunnel-agent@0.4.2
├─┬ service-runner@0.3.9
│ ├── bunyan-syslog-udp@0.1.0
│ ├── core-js@2.1.0
│ ├─┬ gelf-stream@1.1.0
│ │ └── gelfling@0.3.0
│ ├── hot-shots@2.3.1
│ ├── node-txstatsd@0.1.6
│ └─┬ yargs@3.32.0
│   ├── camelcase@2.1.0
│   ├─┬ cliui@3.1.0
│   │ ├─┬ strip-ansi@3.0.0
│   │ │ └── ansi-regex@2.0.0
│   │ └── wrap-ansi@1.0.0
│   ├─┬ decamelize@1.1.2
│   │ └── escape-string-regexp@1.0.4
│   ├─┬ os-locale@1.4.0
│   │ └─┬ lcid@1.0.0
│   │   └── invert-kv@1.0.0
│   ├─┬ string-width@1.0.1
│   │ ├─┬ code-point-at@1.0.0
│   │ │ └── number-is-nan@1.0.0
│   │ └── is-fullwidth-code-point@1.0.0
│   ├── window-size@0.1.4
│   └── y18n@3.2.0
├─┬ swagger-router@0.3.4
│ ├── tassembly@0.2.3
│ └── template-expression-compiler@0.1.8
├── underscore@1.8.3
├─┬ vega@2.4.2
│ ├── canvas@1.3.10
│ ├── d3@3.5.15
│ ├─┬ d3-cloud@1.2.1
│ │ └── d3-dispatch@0.2.6
│ ├─┬ d3-geo-projection@0.2.16
│ │ └─┬ brfs@1.4.3
│ │   ├─┬ quote-stream@1.0.2
│ │   │ ├── buffer-equal@0.0.1
│ │   │ └── minimist@1.2.0
│ │   ├─┬ static-module@1.3.0
│ │   │ ├─┬ duplexer2@0.0.2
│ │   │ │ └── readable-stream@1.1.13
│ │   │ ├─┬ escodegen@1.3.3
│ │   │ │ ├── esprima@1.1.1
│ │   │ │ ├── estraverse@1.5.1
│ │   │ │ ├── esutils@1.0.0
│ │   │ │ └── source-map@0.1.43
│ │   │ ├─┬ falafel@1.2.0
│ │   │ │ ├── acorn@1.2.2
│ │   │ │ ├── foreach@2.0.5
│ │   │ │ └── object-keys@1.0.9
│ │   │ ├─┬ has@1.0.1
│ │   │ │ └── function-bind@1.0.2
│ │   │ ├── object-inspect@0.4.0
│ │   │ ├── quote-stream@0.0.0
│ │   │ ├── readable-stream@1.0.33
│ │   │ ├── shallow-copy@0.0.1
│ │   │ ├─┬ static-eval@0.2.4
│ │   │ │ └─┬ escodegen@0.0.28
│ │   │ │   ├── esprima@1.0.4
│ │   │ │   └── estraverse@1.3.2
│ │   │ └─┬ through2@0.4.2
│ │   │   └─┬ xtend@2.1.2
│ │   │     └── object-keys@0.4.0
│ │   └── through2@2.0.1
│ ├─┬ datalib@1.5.9
│ │ ├── d3-dsv@0.1.14
│ │ ├── d3-format@0.4.2
│ │ ├── d3-time@0.1.1
│ │ ├── d3-time-format@0.2.1
│ │ └─┬ sync-request@2.2.0
│ │   ├─┬ concat-stream@1.4.10
│ │   │ ├── readable-stream@1.1.13
│ │   │ └── typedarray@0.0.6
│ │   ├── http-response-object@1.1.0
│ │   ├─┬ spawn-sync@1.0.15
│ │   │ └── os-shim@0.1.3
│ │   └─┬ then-request@2.1.1
│ │     ├── caseless@0.9.0
│ │     ├─┬ http-basic@2.4.0
│ │     │ └── caseless@0.9.0
│ │     ├─┬ promise@6.1.0
│ │     │ └── asap@1.0.0
│ │     └── qs@2.4.2
│ ├─┬ topojson@1.6.24
│ │ ├── d3-queue@2.0.2
│ │ ├── optimist@0.3.7
│ │ ├── rw@1.1.1
│ │ └─┬ shapefile@0.3.1
│ │   ├── d3-queue@1.2.3
│ │   └── iconv-lite@0.2.11
│ ├── vega-dataflow@1.4.3
│ ├── vega-expression@1.2.0
│ ├── vega-logging@1.0.2
│ └── vega-scenegraph@1.0.17
└─┬ vega-1x@1.5.3 (git+ssh://git@github.com/nyurik/vega-1x.git#79140356950c061ae4abebc3c4cb1f08a4993810)
  ├── d3.layout.cloud@1.2.0
  └─┬ yargs@2.3.0
    └── wordwrap@0.0.2

@Yurik the crucial dependencies are not listed there. The problem are the native c bindings using https://github.com/nodejs/node-gyp

The node package that requires special packages to be installed on the OS is
https://www.npmjs.com/package/librsvg
for mathoid.
For graphoid, the equivalent would probably be
https://www.npmjs.com/package/canvas
.
To which degree the requirements of librsvg and canvans overlap is hard to judge for me.

ok, it seems like this issue is unrelated to getting graphoid to run in Jenkins, as it requires a different dependency. Graphoid relies on "libcairo2-dev", "libgif-dev", "libpango1.0-dev". Additionally, depending on Ubuntu or Debian, it needs "libjpeg62-dev" or "libjpeg62-turbo-dev".

@hashar Is there any information missing to make progress with this task.
It loks like installing librsvg2-dev on CI slaves is all that needs to be done.
Can you give me a pointer to the repo where the mathoid class should go into.
I do not understand the dev_packages option, since this will be a hard requirement to run mathoid.

In T128280#2072150, @hashar wrote:

Graphoid needs a few -dev packages to be provisioned as well ( T119693 ). From that task,the easiest (CI wise) is to have a puppet classe to ship packages required at runtime (ex: libcairo) and optionally the dev packages (ex: libcairo-dev). For example a class mathoid::packages which would take an option such as dev_packages => false which we can then set to True for CI instances.

The idea is to have the list of dependencies listed somewhere out of CI (i.e. in puppet.git) and at a place where people will remember to mention the needed -dev packages.
...

So some puppet work is needed. I have no idea whether apt-get install can be told to also install -dev related packages. And to rephrase my quote above: either have a fooiod::packages::dev or have the foooid::packages to accept a boolean to get the dev packages included.

Then when provision CI nodes we will just have to include the relevant class / with appropriate parameters if needed.

I don't necessarily think having a separate puppet class for packages is a good way to go (tying CI even more with the production infra will only bite us in the long term), but I acknowledge the pragmatism in this workaround, as this is blocking now two projects. I'll refactor the puppet classes slightly so as to support this.

• mobrovac claimed this task.Mar 1 2016, 1:47 PM

• mobrovac triaged this task as High priority.

Macro wikilove: This is a motivating meme to support the task at hand

Change 274675 had a related patch set uploaded (by Mobrovac):
Services: introduce service::packages

https://gerrit.wikimedia.org/r/274675

gerritbot added a project: Patch-For-Review.Mar 3 2016, 11:39 AM

Kind of blocked on T128846: DIB images have /puppet/hieradata files that are empty . Random discussion is happening on https://gerrit.wikimedia.org/r/#/c/274675/

hashar moved this task from Backlog to In progress on the Continuous-Integration-Config board.Mar 8 2016, 1:55 PM

Change 274675 merged by Alexandros Kosiaris:
Services: introduce service::packages

https://gerrit.wikimedia.org/r/274675

hashar added a subtask: T129092: Hiera is not properly configured on Nodepool instances.Mar 9 2016, 10:08 PM

hashar closed subtask T129092: Hiera is not properly configured on Nodepool instances as Resolved.Mar 10 2016, 12:11 PM

Change 276443 had a related patch set uploaded (by Hashar):
dib: include graphoid/mathoid package dependencies

https://gerrit.wikimedia.org/r/276443

Change 276443 merged by jenkins-bot:
dib: include graphoid/mathoid package dependencies

https://gerrit.wikimedia.org/r/276443

Mentioned in SAL [2016-03-10T12:18:13Z] <hashar> Nodepool: rebuilding image to get mathoid/graphoid packages included (hopefully) T119693 T128280

From a puppet run:

Notice: /Stage[main]/Packages::Librsvg2_2/Package[librsvg2-2]/ensure: ensure changed 'purged' to 'present'
Notice: /Stage[main]/Packages::Librsvg2_dev/Package[librsvg2-dev]/ensure: ensure changed 'purged' to 'present'
...

Aced by @mobrovac \O/