Page MenuHomePhabricator

Enable migration code to run on the grid
Closed, ResolvedPublic

Description

When running our code on Toolforge it should be done by submitting it to the grid.

Investigate any blockers to doing this and document how this should be done.

The jobs are run from the same directory as before (but without entering the virtual environment) using e.g. jsub -once -cwd -N IrFamatches /data/project/coh/coh_tools/bin/python3 IrFa.py --list_matches where:

  • -N IrFamatches gives the name of the job.
  • /data/project/coh/coh_tools/bin/python3 ensures that its the virtual environment python client which is run

To check if the code is still running use qstat. Error/output files use the name of the job and by default these end up in the home directory (with .err, .out file endings)
To stop a job use jstop job_name e.g. jstop IrFamatches.

To make this easier, note the name of the job in the relevant upload task :)

Event Timeline

I identified at least one issue (PR up soon) and have code running on the grid right now.