[SOLVED] Slurm Batch Script Save to Variable or Append

Issue

I would like to be able to append my slurm batch scripts to my error file. I submit these on the HPC using sbatch -J "my_run_name" runscript.slurm. The content of runscript.slurm is summarized below.

#!/bin/bash

#SBATCH --job-name=%x
#SBATCH --output=%x.o-%j
#SBATCH --error=%x.e-%j
## Other directives...

error_file="$SLURM_SUBMIT_DIR/$SLURM_JOB_NAME.e-$SLURM_JOB_ID"
main="/path/to/main/file"

# set parameters, then call main function
$main --opt1 $opt1 --opt2 $opt2

# Last line of script
scontrol write batch_script $SLURM_JOB_ID $error_file

Unfortunately this overwrites all the other content I've accumulated in my error_file throughout my long running calculations (lots of log statements on progress) with runscript.slurm

I'm wondering if it is possible to append runscript.slurm to error_file rather than overwrite all existing content in error_file.

On a related point - is it possible to dump the contents of runscript.slurm into a bash variable which I can then pass to my main function (that way I can save runscript.slurm as an additional field in the calculation result). Hopefully that question makes sense.

Solution

Try with

scontrol write batch_script $SLURM_JOB_ID - >> $error_file

Instead of giving SLURM the file where to store the script, you can take its output and redirect it to the file of interest in append mode.

Answered By - Poshi

Answer Checked By - Terry (WPSolving Volunteer)