Page MenuHomePhabricator

libpng problems on deployment-imagescaler02
Closed, ResolvedPublic

Description

I briefly pointed ms-fe towards imagescaler02 instead of 03, and saw this:

Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]: 2019-04-25 17:23:31,822 8801 thumbor:ERROR CommandError: (['/usr/bin/xvfb-run', '-a', '-n', '2', '-s', '-ac -screen 0 1280x1024x24', '/srv/deployment/3d2png/deploy/src/3d2png.js', '/srv/thumbor/tmp/thumbor@8801/tmpPXsW0u', '120x90', '/srv/thumbor/tmp/thumbor@8801/tmpthumb7dYoDX.stl.png'], 'module.js:597\n  return process.dlopen(module, path._makeLong(filename));\n                 ^\n\nError: libpng12.so.0: cannot open shared object file: No such file or directory\n    at Error (native)\n    at Object.Module._extensions..node (module.js:597:18)\n    at Module.load (module.js:487:32)\n    at tryModuleLoad (module.js:446:12)\n    at Function.Module._load (module.js:438:3)\n    at Module.require (module.js:497:17)\n    at require (internal/module.js:20:19)\n    at Object.<anonymous> (/srv/deployment/3d2png/deploy-cache/revs/7fbaaf607686c9063ccb74271bc6a1de794f6a16/node_modules/canvas/lib/bindings.js:3:18)\n    at Module._compile (module.js:570:32)\n    at Object.Module._extensions..js (module.js:579:10)\n', '', 1)
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]: 2019-04-25 17:23:31,823 8801 thumbor:ERROR ERROR: Traceback (most recent call last):
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:   File "/usr/lib/python2.7/dist-packages/thumbor/handlers/__init__.py", line 140, in get_image
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:     self.context.request.image_url
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:   File "/usr/lib/python2.7/dist-packages/tornado/gen.py", line 1015, in run
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:     value = future.result()
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:   File "/usr/lib/python2.7/dist-packages/tornado/concurrent.py", line 237, in result
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:     raise_exc_info(self._exc_info)
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:   File "/usr/lib/python2.7/dist-packages/tornado/gen.py", line 1024, in run
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:     yielded = self.gen.send(value)
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:   File "/usr/lib/python2.7/dist-packages/thumbor/handlers/__init__.py", line 609, in _fetch
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:     self.context.request.engine.load(fetch_result.buffer, extension)
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:   File "/usr/lib/python2.7/dist-packages/wikimedia_thumbor/engine/proxy/proxy.py", line 119, in load
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:     self.lcl[enginename].load(buffer, extension)
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:   File "/usr/lib/python2.7/dist-packages/thumbor/engines/__init__.py", line 167, in load
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:     image_or_frames = self.create_image(buffer)
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:   File "/usr/lib/python2.7/dist-packages/wikimedia_thumbor/engine/stl/stl.py", line 105, in create_image
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]:     raise e
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]: CommandError: (['/usr/bin/xvfb-run', '-a', '-n', '2', '-s', '-ac -screen 0 1280x1024x24', '/srv/deployment/3d2png/deploy/src/3d2png.js', '/srv/thumbor/tmp/thumbor@8801/tmpPXsW0u', '120x90', '/srv/thumbor/tmp/thumbor@8801/tmpthumb7dYoDX.stl.png'], 'module.js:597\n  return process.dlopen(module, path._makeLong(filename));\n                 ^\n\nError: libpng12.so.0: cannot open shared object file: No such file or directory\n    at Error (native)\n    at Object.Module._extensions..node (module.js:597:18)\n    at Module.load (module.js:487:32)\n    at tryModuleLoad (module.js:446:12)\n    at Function.Module._load (module.js:438:3)\n    at Module.require (module.js:497:17)\n    at require (internal/module.js:20:19)\n    at Object.<anonymous> (/srv/deployment/3d2png/deploy-cache/revs/7fbaaf607686c9063ccb74271bc6a1de794f6a16/node_modules/canvas/lib/bindings.js:3:18)\n    at Module._compile (module.js:570:32)\n    at Object.Module._extensions..js (module.js:579:10)\n', '', 1)
Apr 25 17:23:31 deployment-imagescaler02 haproxy[19395]: 172.16.5.163:55794 [25/Apr/2019:17:23:27.173] thumbor thumbor/server8801 0/0/0/4652/4652 500 539 - - ---- 0/0/0/0/0 0/0 "GET /wikipedia/commons/thumb/2/2d/Crystal-1479.stl/120px-Crystal-1479.stl.png HTTP/1.1"
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]: 2019-04-25 17:23:31,823 8801 thumbor:ERROR [BaseHandler] get_image failed for url `http%3A//deployment-ms-fe03.deployment-prep.eqiad.wmflabs/v1/AUTH_mw/wikipedia-commons-local-public.2d/2/2d/Crystal-1479.stl`. error: `(['/usr/bin/xvfb-run', '-a', '-n', '2', '-s', '-ac -screen 0 1280x1024x24', '/srv/deployment/3d2png/deploy/src/3d2png.js', '/srv/thumbor/tmp/thumbor@8801/tmpPXsW0u', '120x90', '/srv/thumbor/tmp/thumbor@8801/tmpthumb7dYoDX.stl.png'], 'module.js:597\n  return process.dlopen(module, path._makeLong(filename));\n                 ^\n\nError: libpng12.so.0: cannot open shared object file: No such file or directory\n    at Error (native)\n    at Object.Module._extensions..node (module.js:597:18)\n    at Module.load (module.js:487:32)\n    at tryModuleLoad (module.js:446:12)\n    at Function.Module._load (module.js:438:3)\n    at Module.require (module.js:497:17)\n    at require (internal/module.js:20:19)\n    at Object.<anonymous> (/srv/deployment/3d2png/deploy-cache/revs/7fbaaf607686c9063ccb74271bc6a1de794f6a16/node_modules/canvas/lib/bindings.js:3:18)\n    at Module._compile (module.js:570:32)\n    at Object.Module._extensions..js (module.js:579:10)\n', '', 1)`
Apr 25 17:23:31 deployment-imagescaler02 thumbor@8801[15747]: 2019-04-25 17:23:31,826 8801 tornado.access:ERROR 500 GET /wikipedia/commons/thumb/2/2d/Crystal-1479.stl/120px-Crystal-1479.stl.png (172.16.5.163) 4651.20ms

key part is probably Error: libpng12.so.0: cannot open shared object file: No such file or directory

Event Timeline

I'm going to guess that this host has an old version of 3d2png installed with node binaries compiled against Jessie instead of Stretch. libpng12-0 doesn't exist in Stretch: https://packages.debian.org/jessie/libpng12-0

Change 506968 had a related patch set uploaded (by Gilles; owner: Gilles):
[3d2png/deploy@master] Update beta cluster targets

https://gerrit.wikimedia.org/r/506968

Change 506968 merged by Gilles:
[3d2png/deploy@master] Update beta cluster targets

https://gerrit.wikimedia.org/r/506968

Gilles claimed this task.

I've deployed the Stretch version of 3d2png on both hosts, and that file renders fine.