Page MenuHomePhabricator

Allow Fish scripts to run on Grid Engine on Toolforge
Closed, ResolvedPublicBUG REPORT

Description

I have some Fish scripts for my Toolforge tool that I'd like to be able to run on the Grid Engine, but fish isn't available there, even though it's available to the scripts when they're run in the console for the tool, outside the Grid Engine. Could Fish be enabled on the Grid Engine?

I posted on IRC in #wikimedia-cloud and someone recommended that I post here.

Demonstration on the console for my tool (after become <my tool name>):

$ which fish
/usr/bin/fish
$ cd ~ && echo $'#! /usr/bin/env fish\n\necho "Hello, world!"' > hello.fish && chmod +x hello.fish
$ /usr/bin/env fish hello.fish
Hello, world!
$ jsub -stderr -sync y hello.fish; tail hello.err
Job xxxxxxx exited with exit code 127.
Your job xxxxxxx ("hello") has been submitted
/usr/bin/env: ‘fish’: No such file or directory

I tried copying /usr/bin/fish locally and running it in the job engine, but it looks like it can't find a PCRE2 shared library that it requires:

$ cd ~ && cp $(which fish) . && echo '#! '$(readlink -f ./fish)$'\n\necho "Hello, world!"' > hello.fish && chmod +x hello.fish
$ jsub hello.fish

After the job runs, hello.err contains /.../fish: error while loading shared libraries: libpcre2-32.so.0: cannot open shared object file: No such file or directory. I thought maybe this was because PCRE2 is too potentially pathological for the Grid Engine. (But I'd be surprised if other languages could get along without it.)

Is this intentional or could Fish be enabled on the Grid Engine? I suppose I could try to rewrite my scripts in another language if necessary.

Event Timeline

I tried copying /usr/bin/fish locally and running it in the job engine,

This doesn't work because of dynamic linking.

Fish is supposed to be installed by T219054: Install fish shell for Toolforge use, but it was only for bastions. Just to confirm, this request for fish to be installed to the grid exec nodes right?

Just to confirm, this request for fish to be installed to the grid exec nodes right?

Probably, but I am new to Toolforge and do not understand it very well. My request is for Fish scripts to be able to run when I submit them as jobs with jsub.

Erutuon renamed this task from Can't run Fish scripts on Grid Engine on Toolforge to Allow Fish scripts to run on Grid Engine on Toolforge.Dec 21 2019, 1:27 AM
Erutuon updated the task description. (Show Details)

Change 560008 had a related patch set uploaded (by Zhuyifei1999; owner: Zhuyifei1999):
[operations/puppet@production] toolforge: Move package 'fish' from shell_environ to exec_environ

https://gerrit.wikimedia.org/r/560008

Change 560008 merged by Andrew Bogott:
[operations/puppet@production] toolforge: Move package 'fish' from shell_environ to exec_environ

https://gerrit.wikimedia.org/r/560008

zhuyifei1999 claimed this task.