Page MenuHomePhabricator

Node.js on gridengine
Closed, ResolvedPublic

Description

I'm trying to run my NodeJS script on the grid engine. I see a very old of node (v0.10.25) is installed globally at /usr/bin/node which I cannot use. I have downloaded a recent version (v6.10.3) that is able to run my program fine:

/data/project/congressedits/node-v6.10.3-linux-x64/bin/node

But when I try to start it in the grid environment with this:

jstart -N congresseditors /data/project/congressedits/node/bin/node /data/project/congressedits/congresseditors/congresseditors.js --config /data/project/congressedits/congresseditors/config.json --verbose

I get this in congresseditors.err

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [/mnt/nfs/labstore-secondary-tools-project/congressedits/node-v6.10.3-linux-x64/bin/node]
2: 0x109b9cc [/mnt/nfs/labstore-secondary-tools-project/congressedits/node-v6.10.3-linux-x64/bin/node]
3: v8::Utils::ReportApiFailure(char const*, char const*) [/mnt/nfs/labstore-secondary-tools-project/congressedits/node-v6.10.3-linux-x64/bin/node]
4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/mnt/nfs/labstore-secondary-tools-project/congressedits/node-v6.10.3-linux-x64/bin/node]
5: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/mnt/nfs/labstore-secondary-tools-project/congressedits/node-v6.10.3-linux-x64/bin/node]
6: v8::internal::Factory::NewStringFromUtf8(v8::internal::Vector<char const>, v8::internal::PretenureFlag) [/mnt/nfs/labstore-secondary-tools-project/congressedits/node-v6.10.3-linux-x64/bin/node]
7: v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::String::NewStringType, int) [/mnt/nfs/labstore-secondary-tools-project/congressedits/node-v6.10.3-linux-x64/bin/node]
8: node::StringBytes::Encode(v8::Isolate*, char const*, unsigned long, node::encoding) [/mnt/nfs/labstore-secondary-tools-project/congressedits/node-v6.10.3-linux-x64/bin/node]
9: void node::Buffer::StringSlice<(node::encoding)1>(v8::FunctionCallbackInfo<v8::Value> const&) [/mnt/nfs/labstore-secondary-tools-project/congressedits/node-v6.10.3-linux-x64/bin/node]
10: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) [/mnt/nfs/labstore-secondary-tools-project/congressedits/node-v6.10.3-linux-x64/bin/node]
11: 0x9ec36e [/mnt/nfs/labstore-secondary-tools-project/congressedits/node-v6.10.3-linux-x64/bin/node]
12: 0x9ecc0e [/mnt/nfs/labstore-secondary-tools-project/congressedits/node-v6.10.3-linux-x64/bin/node]
13: 0x27b5273092a7
Aborted (core dumped)

And this in congresseditors.out

<--- Last few GCs --->

3404 ms: Mark-sweep 21.8 (39.0) -> 15.5 (39.0) MB, 33.1 / 0.0 ms [allocation failure] [GC in old space requested].
3431 ms: Mark-sweep 15.5 (39.0) -> 15.4 (39.0) MB, 26.1 / 0.0 ms [allocation failure] [GC in old space requested].
3454 ms: Mark-sweep 15.4 (39.0) -> 14.1 (24.0) MB, 23.2 / 0.0 ms [last resort gc].
3493 ms: Mark-sweep 14.1 (24.0) -> 15.2 (22.0) MB, 38.8 / 0.0 ms [last resort gc].

<--- JS stacktrace --->

JS stack trace

Security context: 0x18681a5cfb39 <JS Object>

1: slowToString [buffer.js:459] [pc=0x27b52776f215] (this=0x3c72aa0a9b39 <an Uint8Array with map 0x37ea646068d9>,encoding=0x18681a5dd309 <String[4]: utf8>,start=0,end=720427)
2: arguments adaptor frame: 1->3
3: toString [buffer.js:492] [pc=0x27b52776e9df] (this=0x3c72aa0a9b39 <an Uint8Array with map 0x37ea646068d9>)
4: arguments adaptor frame: 1->0
5: /* anonymous */ [/mnt/n...

Is there a better way of doing this?

Event Timeline

Krinkle renamed this task from node on gridengine to Tool Labs node on gridengine.Jul 7 2017, 10:33 PM
Krinkle edited projects, added Toolforge; removed Tool-Global-user-contributions.
Luke081515 renamed this task from Tool Labs node on gridengine to Node.js on gridengine.Jul 7 2017, 10:41 PM

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

You likely need to request a larger amount of memory using the -mem parameter to jstart.

I actually have tried upping the memory. It just takes longer to run out of memory. I would like to upgrade Node if possible.

It just takes longer to run out of memory.

Memory leak? There will be an upper limit on the memory that are available to you anyhow.

I would like to upgrade Node if possible.

Try running node on kubernetes (which has a system node version of v6.9.1), or you can install a newer version of Node via nvm or other methods.

The memory leak is a result of using the old version of node with the Cheerio library. If node can be upgraded that would help me. v0.10 is end-of-life and is no longer maintained.

If node can be upgraded that would help me. v0.10 is end-of-life and is no longer maintained.

I would like to upgrade Node if possible.

Try running node on kubernetes (which has a system node version of v6.9.1), or you can install a newer version of Node via nvm or other methods.

See https://wikitech.wikimedia.org/wiki/Help:Toolforge/Kubernetes

Btw, regarding EOL: because this is installed from ubuntu trusty apt, ubuntu will backport any security updates until trusty's EOL.

Upgrading nodejs on Trusty hosts has been discussed before, but I'm not easily finding the phab tickets where it happened. the TL;DR is that we deploy the nodejs packages found in either the upstream vendor apt repo or the Wikimedia Foundation managed apt repo (https://apt.wikimedia.org/wikimedia/). The Cloud Services team does not currently have the resources that would be needed to maintain custom packages that deviate from these sources.

On Debian Jessie systems the Foundation is tracking long term stable nodejs branches and currently using v6.11 (or v6.9 on systems waiting to be updated T170716). In Toolforge, grid engine uses Ubuntu Trusty hosts and Kubernetes uses Debian Jessie. Due to the ancient version of nodejs provided for Trusty, we recommend using Kubernetes pods for nodejs tools. See https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web for webservice instructions and https://wikitech.wikimedia.org/wiki/Help:Toolforge/Kubernetes for less polished instructions on running scheduled or continuous jobs.

Ok, thanks for the details, this helps a lot. So it sounds like the way to move forward is to start using Kubernetes which I wanted to learn about anyway, and now I have a reason.

I'm a bit confused about how to get npm install to run under kubernetes with the docker-registry.tools.wmflabs.org/toollabs-nodejs-base:latest image. Is there an example of that somewhere?

bd808 claimed this task.

You can get a shell using webservice --backend=kubernetes nodejs shell. This will start a Docker container on the Kubernetes cluster, mount your tool's directories, and give you control of a /bin/bash shell session there. I have added basic documentation for this on wikitech: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web#Running_npm_with_webservice_shell