Page MenuHomePhabricator

Enable librsvg-dev on jenkins
Closed, ResolvedPublic

Description

Tests fail for mathoid-mathjax-node@0.6.0 cf. https://gerrit.wikimedia.org/r/#/c/270547/

15:42:57 > librsvg@0.6.1 install /home/jenkins/workspace/npm-node-4.3/node_modules/mathoid-mathjax-node/node_modules/librsvg
15:42:57 > node-gyp rebuild
15:42:57 
15:42:57 (node) child_process: options.customFds option is deprecated. Use options.stdio instead.
15:42:57 Package librsvg-2.0 was not found in the pkg-config search path.
15:42:57 Perhaps you should add the directory containing `librsvg-2.0.pc'
15:42:57 to the PKG_CONFIG_PATH environment variable
15:42:57 No package 'librsvg-2.0' found
15:42:57 Package cairo-png was not found in the pkg-config search path.
15:42:57 Perhaps you should add the directory containing `cairo-png.pc'
15:42:57 to the PKG_CONFIG_PATH environment variable
15:42:57 No package 'cairo-png' found
15:42:57 Package cairo-pdf was not found in the pkg-config search path.
15:42:57 Perhaps you should add the directory containing `cairo-pdf.pc'
15:42:57 to the PKG_CONFIG_PATH environment variable
15:42:57 No package 'cairo-pdf' found
15:42:57 Package cairo-svg was not found in the pkg-config search path.
15:42:57 Perhaps you should add the directory containing `cairo-svg.pc'
15:42:57 to the PKG_CONFIG_PATH environment variable
15:42:57 No package 'cairo-svg' found
15:42:57 gyp: Call to 'pkg-config --libs-only-l librsvg-2.0 cairo-png cairo-pdf cairo-svg' returned exit status 1 while in binding.gyp.
15:42:57 gyp ERR! configure error 
15:42:57 gyp ERR! stack Error: `gyp` failed with exit code: 1
15:42:57 gyp ERR! stack     at ChildProcess.onCpExit (/usr/share/node-gyp/lib/configure.js:344:16)
15:42:57 gyp ERR! stack     at emitTwo (events.js:87:13)
15:42:57 gyp ERR! stack     at ChildProcess.emit (events.js:172:7)
15:42:57 gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)
15:42:57 gyp ERR! System Linux 3.16.0-4-amd64
15:42:57 gyp ERR! command "/usr/bin/nodejs" "/usr/bin/node-gyp" "rebuild"
15:42:57 gyp ERR! cwd /home/jenkins/workspace/npm-node-4.3/node_modules/mathoid-mathjax-node/node_modules/librsvg
15:42:57 gyp ERR! node -v v4.3.0
15:42:57 gyp ERR! node-gyp -v v0.12.2
15:42:57 gyp ERR! not ok 
15:42:57 npm WARN This failure might be due to the use of legacy binary "node"
15:42:57 npm WARN For further explanations, please read
15:42:57 /usr/share/doc/nodejs/README.Debian

Event Timeline

Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptFeb 27 2016, 3:46 PM
Physikerwelt updated the task description. (Show Details)Feb 27 2016, 3:47 PM

Graphoid needs a few -dev packages to be provisioned as well ( T119693 ). From that task,the easiest (CI wise) is to have a puppet classe to ship packages required at runtime (ex: libcairo) and optionally the dev packages (ex: libcairo-dev). For example a class mathoid::packages which would take an option such as dev_packages => false which we can then set to True for CI instances.

The idea is to have the list of dependencies listed somewhere out of CI (i.e. in puppet.git) and at a place where people will remember to mention the needed -dev packages.

But maybe there is better way that would match both Mathoid and Graphoid use cases.

on my development instance (running ubuntu 14 LTS) everything I did was running

sudo apt-get install librsvg2-dev

(http://packages.ubuntu.com/de/trusty/librsvg2-dev)
What would be the data that I as a Jenkins user have to provide in that case.

Yurik added a comment.Feb 29 2016, 6:28 PM

Graphoid dependencies tree, just in case it is needed.

yurik@steppenwolf:~/wmf/graphoid/graphoid$ npm ls
graphoid@0.1.7 /home/yurik/wmf/graphoid/graphoid
├── bluebird@3.3.1
├─┬ body-parser@1.15.0
│ ├── bytes@2.2.0
│ ├── content-type@1.0.1
│ ├─┬ debug@2.2.0
│ │ └── ms@0.7.1
│ ├── depd@1.1.0
│ ├─┬ http-errors@1.4.0
│ │ ├── inherits@2.0.1
│ │ └── statuses@1.2.1
│ ├── iconv-lite@0.4.13
│ ├─┬ on-finished@2.3.0
│ │ └── ee-first@1.1.1
│ ├── qs@6.1.0
│ ├─┬ raw-body@2.1.5
│ │ └── unpipe@1.0.0
│ └─┬ type-is@1.6.11
│   ├── media-typer@0.3.0
│   └── mime-types@2.1.9
├─┬ bunyan@1.6.0
│ ├─┬ dtrace-provider@0.6.0
│ │ └── nan@2.2.0
│ ├── moment@2.11.2
│ ├─┬ mv@2.1.1
│ │ ├── ncp@2.0.0
│ │ └─┬ rimraf@2.4.5
│ │   └── glob@6.0.4
│ └── safe-json-stringify@1.0.3
├─┬ cassandra-uuid@0.0.2
│ └── long@2.4.0
├─┬ compression@1.6.1
│ ├─┬ accepts@1.3.1
│ │ └── negotiator@0.6.0
│ ├─┬ compressible@2.0.7
│ │ └── mime-db@1.21.0
│ ├── on-headers@1.0.1
│ └── vary@1.1.0
├── domino@1.0.23
├─┬ express@4.13.4
│ ├─┬ accepts@1.2.13
│ │ └── negotiator@0.5.3
│ ├── array-flatten@1.1.1
│ ├── content-disposition@0.5.1
│ ├── cookie@0.1.5
│ ├── cookie-signature@1.0.6
│ ├── escape-html@1.0.3
│ ├── etag@1.7.0
│ ├── finalhandler@0.4.1
│ ├── fresh@0.3.0
│ ├── merge-descriptors@1.0.1
│ ├── methods@1.1.2
│ ├── parseurl@1.3.1
│ ├── path-to-regexp@0.1.7
│ ├─┬ proxy-addr@1.0.10
│ │ ├── forwarded@0.1.0
│ │ └── ipaddr.js@1.0.5
│ ├── qs@4.0.0
│ ├── range-parser@1.0.3
│ ├─┬ send@0.13.1
│ │ ├── destroy@1.0.4
│ │ ├── http-errors@1.3.1
│ │ └── mime@1.3.4
│ ├── serve-static@1.10.2
│ ├── utils-merge@1.0.0
│ └── vary@1.0.1
├── extend@3.0.0
├── graph-shared@0.0.2 (git+ssh://git@github.com/nyurik/graph-shared.git#852742aba9206e451e42383361978cbccaaeaad2)
├─┬ istanbul@0.4.2
│ ├── abbrev@1.0.7
│ ├── async@1.5.2
│ ├─┬ escodegen@1.7.1
│ │ ├── esprima@1.2.5
│ │ ├── estraverse@1.9.3
│ │ ├── esutils@2.0.2
│ │ ├─┬ optionator@0.5.0
│ │ │ ├── deep-is@0.1.3
│ │ │ ├── fast-levenshtein@1.0.7
│ │ │ ├── levn@0.2.5
│ │ │ ├── prelude-ls@1.1.2
│ │ │ └── type-check@0.3.2
│ │ └── source-map@0.2.0
│ ├── esprima@2.7.2
│ ├─┬ fileset@0.2.1
│ │ ├─┬ glob@5.0.15
│ │ │ ├── inflight@1.0.4
│ │ │ └── path-is-absolute@1.0.0
│ │ └── minimatch@2.0.10
│ ├─┬ handlebars@4.0.5
│ │ ├─┬ optimist@0.6.1
│ │ │ └── wordwrap@0.0.3
│ │ ├─┬ source-map@0.4.4
│ │ │ └── amdefine@1.0.0
│ │ └─┬ uglify-js@2.6.1
│ │   ├── async@0.2.10
│ │   ├── source-map@0.5.3
│ │   ├── uglify-to-browserify@1.0.2
│ │   └─┬ yargs@3.10.0
│ │     ├── camelcase@1.2.1
│ │     ├─┬ cliui@2.1.0
│ │     │ ├─┬ center-align@0.1.3
│ │     │ │ ├─┬ align-text@0.1.4
│ │     │ │ │ ├─┬ kind-of@3.0.2
│ │     │ │ │ │ └── is-buffer@1.1.2
│ │     │ │ │ ├── longest@1.0.1
│ │     │ │ │ └── repeat-string@1.5.2
│ │     │ │ └── lazy-cache@1.0.3
│ │     │ ├── right-align@0.1.3
│ │     │ └── wordwrap@0.0.2
│ │     └── window-size@0.1.0
│ ├── nopt@3.0.6
│ ├─┬ once@1.3.3
│ │ └── wrappy@1.0.1
│ ├── resolve@1.1.7
│ ├─┬ supports-color@3.1.2
│ │ └── has-flag@1.0.0
│ ├─┬ which@1.2.4
│ │ ├─┬ is-absolute@0.1.7
│ │ │ └── is-relative@0.1.3
│ │ └── isexe@1.1.2
│ └── wordwrap@1.0.0
├─┬ js-yaml@3.5.3
│ └─┬ argparse@1.0.6
│   └── sprintf-js@1.0.3
├─┬ mkdirp@0.5.1
│ └── minimist@0.0.8
├─┬ mocha@2.4.5
│ ├── commander@2.3.0
│ ├── diff@1.4.0
│ ├── escape-string-regexp@1.0.2
│ ├─┬ glob@3.2.3
│ │ ├── graceful-fs@2.0.3
│ │ └─┬ minimatch@0.2.14
│ │   ├── lru-cache@2.7.3
│ │   └── sigmund@1.0.1
│ ├── growl@1.8.1
│ ├─┬ jade@0.26.3
│ │ ├── commander@0.6.1
│ │ └── mkdirp@0.3.0
│ └── supports-color@1.2.0
├─┬ mocha-jshint@2.3.1
│ ├─┬ jshint@2.9.1
│ │ ├─┬ cli@0.6.6
│ │ │ └─┬ glob@3.2.11
│ │ │   └── minimatch@0.3.0
│ │ ├─┬ console-browserify@1.1.0
│ │ │ └── date-now@0.1.4
│ │ ├── exit@0.1.2
│ │ ├─┬ htmlparser2@3.8.3
│ │ │ ├── domelementtype@1.3.0
│ │ │ ├── domhandler@2.3.0
│ │ │ ├─┬ domutils@1.5.1
│ │ │ │ └─┬ dom-serializer@0.1.0
│ │ │ │   ├── domelementtype@1.1.3
│ │ │ │   └── entities@1.1.1
│ │ │ ├── entities@1.0.0
│ │ │ └─┬ readable-stream@1.1.13
│ │ │   ├── core-util-is@1.0.2
│ │ │   ├── isarray@0.0.1
│ │ │   └── string_decoder@0.10.31
│ │ ├── lodash@3.7.0
│ │ ├── minimatch@2.0.10
│ │ ├── shelljs@0.3.0
│ │ └── strip-json-comments@1.0.4
│ ├─┬ minimatch@3.0.0
│ │ └─┬ brace-expansion@1.1.3
│ │   ├── balanced-match@0.3.0
│ │   └── concat-map@0.0.1
│ ├── shelljs@0.4.0
│ └── uniq@1.0.1
├── mocha-lcov-reporter@1.0.0
├─┬ preq@0.4.8
│ └─┬ request@2.69.0
│   ├── aws-sign2@0.6.0
│   ├── aws4@1.2.1
│   ├─┬ bl@1.0.3
│   │ └─┬ readable-stream@2.0.5
│   │   ├── process-nextick-args@1.0.6
│   │   └── util-deprecate@1.0.2
│   ├── caseless@0.11.0
│   ├─┬ combined-stream@1.0.5
│   │ └── delayed-stream@1.0.0
│   ├── forever-agent@0.6.1
│   ├── form-data@1.0.0-rc3
│   ├─┬ har-validator@2.0.6
│   │ ├─┬ chalk@1.1.1
│   │ │ ├── ansi-styles@2.1.0
│   │ │ ├── has-ansi@2.0.0
│   │ │ └── supports-color@2.0.0
│   │ ├─┬ commander@2.9.0
│   │ │ └── graceful-readlink@1.0.1
│   │ ├─┬ is-my-json-valid@2.12.4
│   │ │ ├── generate-function@2.0.0
│   │ │ ├─┬ generate-object-property@1.2.0
│   │ │ │ └── is-property@1.0.2
│   │ │ ├── jsonpointer@2.0.0
│   │ │ └── xtend@4.0.1
│   │ └─┬ pinkie-promise@2.0.0
│   │   └── pinkie@2.0.4
│   ├─┬ hawk@3.1.3
│   │ ├── boom@2.10.1
│   │ ├── cryptiles@2.0.5
│   │ ├── hoek@2.16.3
│   │ └── sntp@1.0.9
│   ├─┬ http-signature@1.1.1
│   │ ├── assert-plus@0.2.0
│   │ ├─┬ jsprim@1.2.2
│   │ │ ├── extsprintf@1.0.2
│   │ │ ├── json-schema@0.2.2
│   │ │ └── verror@1.3.6
│   │ └─┬ sshpk@1.7.4
│   │   ├── asn1@0.2.3
│   │   ├── dashdash@1.12.2
│   │   ├── ecc-jsbn@0.1.1
│   │   ├── jodid25519@1.0.2
│   │   ├── jsbn@0.1.0
│   │   └── tweetnacl@0.13.3
│   ├── is-typedarray@1.0.0
│   ├── isstream@0.1.2
│   ├── json-stringify-safe@5.0.1
│   ├── node-uuid@1.4.7
│   ├── oauth-sign@0.8.1
│   ├── qs@6.0.2
│   ├── stringstream@0.0.5
│   ├── tough-cookie@2.2.1
│   └── tunnel-agent@0.4.2
├─┬ service-runner@0.3.9
│ ├── bunyan-syslog-udp@0.1.0
│ ├── core-js@2.1.0
│ ├─┬ gelf-stream@1.1.0
│ │ └── gelfling@0.3.0
│ ├── hot-shots@2.3.1
│ ├── node-txstatsd@0.1.6
│ └─┬ yargs@3.32.0
│   ├── camelcase@2.1.0
│   ├─┬ cliui@3.1.0
│   │ ├─┬ strip-ansi@3.0.0
│   │ │ └── ansi-regex@2.0.0
│   │ └── wrap-ansi@1.0.0
│   ├─┬ decamelize@1.1.2
│   │ └── escape-string-regexp@1.0.4
│   ├─┬ os-locale@1.4.0
│   │ └─┬ lcid@1.0.0
│   │   └── invert-kv@1.0.0
│   ├─┬ string-width@1.0.1
│   │ ├─┬ code-point-at@1.0.0
│   │ │ └── number-is-nan@1.0.0
│   │ └── is-fullwidth-code-point@1.0.0
│   ├── window-size@0.1.4
│   └── y18n@3.2.0
├─┬ swagger-router@0.3.4
│ ├── tassembly@0.2.3
│ └── template-expression-compiler@0.1.8
├── underscore@1.8.3
├─┬ vega@2.4.2
│ ├── canvas@1.3.10
│ ├── d3@3.5.15
│ ├─┬ d3-cloud@1.2.1
│ │ └── d3-dispatch@0.2.6
│ ├─┬ d3-geo-projection@0.2.16
│ │ └─┬ brfs@1.4.3
│ │   ├─┬ quote-stream@1.0.2
│ │   │ ├── buffer-equal@0.0.1
│ │   │ └── minimist@1.2.0
│ │   ├─┬ static-module@1.3.0
│ │   │ ├─┬ duplexer2@0.0.2
│ │   │ │ └── readable-stream@1.1.13
│ │   │ ├─┬ escodegen@1.3.3
│ │   │ │ ├── esprima@1.1.1
│ │   │ │ ├── estraverse@1.5.1
│ │   │ │ ├── esutils@1.0.0
│ │   │ │ └── source-map@0.1.43
│ │   │ ├─┬ falafel@1.2.0
│ │   │ │ ├── acorn@1.2.2
│ │   │ │ ├── foreach@2.0.5
│ │   │ │ └── object-keys@1.0.9
│ │   │ ├─┬ has@1.0.1
│ │   │ │ └── function-bind@1.0.2
│ │   │ ├── object-inspect@0.4.0
│ │   │ ├── quote-stream@0.0.0
│ │   │ ├── readable-stream@1.0.33
│ │   │ ├── shallow-copy@0.0.1
│ │   │ ├─┬ static-eval@0.2.4
│ │   │ │ └─┬ escodegen@0.0.28
│ │   │ │   ├── esprima@1.0.4
│ │   │ │   └── estraverse@1.3.2
│ │   │ └─┬ through2@0.4.2
│ │   │   └─┬ xtend@2.1.2
│ │   │     └── object-keys@0.4.0
│ │   └── through2@2.0.1
│ ├─┬ datalib@1.5.9
│ │ ├── d3-dsv@0.1.14
│ │ ├── d3-format@0.4.2
│ │ ├── d3-time@0.1.1
│ │ ├── d3-time-format@0.2.1
│ │ └─┬ sync-request@2.2.0
│ │   ├─┬ concat-stream@1.4.10
│ │   │ ├── readable-stream@1.1.13
│ │   │ └── typedarray@0.0.6
│ │   ├── http-response-object@1.1.0
│ │   ├─┬ spawn-sync@1.0.15
│ │   │ └── os-shim@0.1.3
│ │   └─┬ then-request@2.1.1
│ │     ├── caseless@0.9.0
│ │     ├─┬ http-basic@2.4.0
│ │     │ └── caseless@0.9.0
│ │     ├─┬ promise@6.1.0
│ │     │ └── asap@1.0.0
│ │     └── qs@2.4.2
│ ├─┬ topojson@1.6.24
│ │ ├── d3-queue@2.0.2
│ │ ├── optimist@0.3.7
│ │ ├── rw@1.1.1
│ │ └─┬ shapefile@0.3.1
│ │   ├── d3-queue@1.2.3
│ │   └── iconv-lite@0.2.11
│ ├── vega-dataflow@1.4.3
│ ├── vega-expression@1.2.0
│ ├── vega-logging@1.0.2
│ └── vega-scenegraph@1.0.17
└─┬ vega-1x@1.5.3 (git+ssh://git@github.com/nyurik/vega-1x.git#79140356950c061ae4abebc3c4cb1f08a4993810)
  ├── d3.layout.cloud@1.2.0
  └─┬ yargs@2.3.0
    └── wordwrap@0.0.2

@Yurik the crucial dependencies are not listed there. The problem are the native c bindings using https://github.com/nodejs/node-gyp

Physikerwelt added a comment.EditedFeb 29 2016, 6:56 PM

The node package that requires special packages to be installed on the OS is
https://www.npmjs.com/package/librsvg
for mathoid.
For graphoid, the equivalent would probably be
https://www.npmjs.com/package/canvas
.
To which degree the requirements of librsvg and canvans overlap is hard to judge for me.

Yurik added a comment.Feb 29 2016, 7:15 PM

ok, it seems like this issue is unrelated to getting graphoid to run in Jenkins, as it requires a different dependency. Graphoid relies on "libcairo2-dev", "libgif-dev", "libpango1.0-dev". Additionally, depending on Ubuntu or Debian, it needs "libjpeg62-dev" or "libjpeg62-turbo-dev".

Physikerwelt added a comment.EditedMar 1 2016, 9:34 AM

@hashar Is there any information missing to make progress with this task.
It loks like installing librsvg2-dev on CI slaves is all that needs to be done.
Can you give me a pointer to the repo where the mathoid class should go into.
I do not understand the dev_packages option, since this will be a hard requirement to run mathoid.

Graphoid needs a few -dev packages to be provisioned as well ( T119693 ). From that task,the easiest (CI wise) is to have a puppet classe to ship packages required at runtime (ex: libcairo) and optionally the dev packages (ex: libcairo-dev). For example a class mathoid::packages which would take an option such as dev_packages => false which we can then set to True for CI instances.

The idea is to have the list of dependencies listed somewhere out of CI (i.e. in puppet.git) and at a place where people will remember to mention the needed -dev packages.
...

So some puppet work is needed. I have no idea whether apt-get install can be told to also install -dev related packages. And to rephrase my quote above: either have a fooiod::packages::dev or have the foooid::packages to accept a boolean to get the dev packages included.

Then when provision CI nodes we will just have to include the relevant class / with appropriate parameters if needed.

I don't necessarily think having a separate puppet class for packages is a good way to go (tying CI even more with the production infra will only bite us in the long term), but I acknowledge the pragmatism in this workaround, as this is blocking now two projects. I'll refactor the puppet classes slightly so as to support this.

mobrovac claimed this task.Mar 1 2016, 1:47 PM
mobrovac triaged this task as High priority.

Change 274675 had a related patch set uploaded (by Mobrovac):
Services: introduce service::packages

https://gerrit.wikimedia.org/r/274675

Change 274675 merged by Alexandros Kosiaris:
Services: introduce service::packages

https://gerrit.wikimedia.org/r/274675

Change 276443 had a related patch set uploaded (by Hashar):
dib: include graphoid/mathoid package dependencies

https://gerrit.wikimedia.org/r/276443

Change 276443 merged by jenkins-bot:
dib: include graphoid/mathoid package dependencies

https://gerrit.wikimedia.org/r/276443

Mentioned in SAL [2016-03-10T12:18:13Z] <hashar> Nodepool: rebuilding image to get mathoid/graphoid packages included (hopefully) T119693 T128280

hashar closed this task as Resolved.Mar 10 2016, 12:26 PM

From a puppet run:

Notice: /Stage[main]/Packages::Librsvg2_2/Package[librsvg2-2]/ensure: ensure changed 'purged' to 'present'
Notice: /Stage[main]/Packages::Librsvg2_dev/Package[librsvg2-dev]/ensure: ensure changed 'purged' to 'present'
...

Aced by @mobrovac \O/

Validated on the Mathoid patch that adds librsvg https://gerrit.wikimedia.org/r/#/c/270547/7

Aklapper added a project: Math.Jun 24 2020, 5:53 AM