Steps to replicate the issue (include links if applicable):
- Become your tool account on bastion
- Create a python environment: python3 -m venv /path/to/env
- Activate the environment: . /path/to/env/bin/activate
- Install Pillow: python3 -m pip install Pillow
- Save the script provided below in a file located at /path/to/script.py
script.py
from PIL import Image
- Save the following script in a file located at /path/to/wrapper.sh
wrapper.sh
. ~/env/bin/activate python3 ~/path/to/script.py deactivate
- First run the wrapper script directly on Bastion: . /path/to/wrapper.sh
- Now create a job on toolforge job framework to run the same wrapper script:
tfj run "testpillow" --command ~/path/to/wrapper.sh -e ~/path/to/wrapper.log -o ~/path/to/wrapper.log --image tf-python37
What happens?:
Running the command on Bastion succeeds.
Running the command as a job results in an error message that is similar to this:
Traceback (most recent call last):
File "/data/project/huji/core/pwb.py", line 39, in <module>
sys.exit(main())
File "/data/project/huji/core/pwb.py", line 35, in main
runpy.run_path(str(path), run_name='__main__')
File "/usr/lib/python3.7/runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "/usr/lib/python3.7/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/data/project/huji/core/pywikibot/scripts/wrapper.py", line 513, in <module>
main()
File "/data/project/huji/core/pywikibot/scripts/wrapper.py", line 497, in main
if not execute():
File "/data/project/huji/core/pywikibot/scripts/wrapper.py", line 484, in execute
run_python_file(filename, script_args, module)
File "/data/project/huji/core/pywikibot/scripts/wrapper.py", line 148, in run_python_file
main_mod.__dict__)
File "/data/project/huji/core/scripts/userscripts/test_pillow.py", line 4, in <module>
from PIL import Image
File "/data/project/huji/env/lib/python3.7/site-packages/PIL/Image.py", line 132, in <module>
from . import _imaging as core
ImportError: libopenjp2.so.7: cannot open shared object file: No such file or directory
CRITICAL: Exiting due to uncaught exception ImportError: libopenjp2.so.7: cannot open shared object file: No such file or directoryWhat should have happened instead?:
The wrapper script should run without errors when executed on toolforge jobs framework
Other information (browser name/version, screenshots, etc.):
Essentially, the Pillow python package is failing to run, due to a missing dependency.
According to Stack Overflow this can be fixed by running sudo apt install libopenjp2-7 on the machine that is running the script.