I know I can set variables in my slurm script. Can I set variables in my #SBATCH line?
For example if I wrote:
#SBATCH --output=/FULL_PATH_TO_SOME_DIR/outfile.txt
Could I obtain the same output if I write instead:
DIRVAR=FULL_PATH_TO_SOME_DIR
#SBATCH --output=/$DIRVAR/outfile.txt
Thanks!
Unfortunately this could not be done in the way you described. The #SBATCH directives are parsed and applied by SLURM job scheduler before the script is executed later during the job execution. At that time the #SBATCH directives will be merely comments to the script interpreter. Short answer, we can’t do that.
Alternative 1:
A way to sidestep this problem is to create a submission script which will submit the job script to the SLURM scheduler with whatever customization needed. Here’s a skeleton example to submit a hypothetical SLURM script called myjob.sh:
#!/bin/bash
# ... other stuff here
DIRVAR=FULL_PATH_TO_SOME_DIR
sbatch --output=/$DIRVAR/outfile.txt myjob.sh
Save this as another local script which generally will not be submitted to SLURM, but rather, will drive the submission(s) of myjob.sh to SLURM.
In this way, when sbatch processes myjob.sh, it is not dealing with any shell variable, since $DIRVAR would have been evaluated before the sbatch command is invoked.
Alternative 2:
iff the variable part of the output name or path has strictly to do with the job number, job name, user name plus fixed substrings, then you may be able to leverage “filename pattern” recognized by SLURM. See filename pattern in SLURM Manual.