Interactive Jobs¶

Why not just run on the login node?¶

When you SSH into Unity, you land on a login node: a shared gateway used by everyone on the cluster simultaneously. It's meant for light tasks: editing files, writing job scripts, submitting jobs, checking queue status.

The login node is not meant for computation. Running a heavy analysis there slows it down for every other user logged in at the same time, and cluster admins actively kill resource-heavy processes on the login node. Even a moderately expensive Python script loading a large dataset can degrade the experience for dozens of colleagues.

The rule of thumb: if a command takes more than a few seconds or uses more than a small amount of RAM, run it on a compute node. Interactive jobs give you a shell on a compute node. You type commands there just like normal, but the resources belong to you exclusively.

srun --pty bash — Getting an interactive shell¶

The simplest way to get an interactive session is with srun:

srun --ntasks=1 --cpus-per-task=4 --mem=8G --time=02:00:00 --pty bash

What each flag does:

After a moment in the queue, your prompt changes to show the name of the compute node you've been assigned (something like [userid@cpu042 ~]$). You're now running on a compute node. Run your commands, inspect your data, test your code. When you're done, type exit to release the allocation and return to the login node.

Requesting a GPU interactively¶

If you need a GPU (to test a model, check CUDA availability, or profile GPU code), add --gpus and specify the GPU partition:

srun --ntasks=1 --cpus-per-task=4 --mem=16G --gpus=1 --partition=gpu --time=01:00:00 --pty bash

For short GPU sessions (under 2 hours), the gpu-preempt partition often has more available hardware and shorter queue times:

srun --ntasks=1 --cpus-per-task=4 --mem=16G --gpus=1 --partition=gpu-preempt -q short --time=01:00:00 --pty bash

The tradeoff: jobs on preempt partitions can be killed after 2 hours by a priority user whose hardware you're borrowing. For quick testing and debugging, that's usually fine.

Once you're connected, verify that the GPU is visible:

nvidia-smi

You should see the GPU model, memory usage, and driver version. If nvidia-smi returns nothing, confirm you're on a node with GPUs using echo $SLURMD_NODENAME and cross-check with sinfo.

If you need a specific GPU type, use --constraint:

srun --ntasks=1 --cpus-per-task=4 --mem=16G --gpus=1 --constraint=a100 --partition=gpu --time=01:00:00 --pty bash

Common constraint values include GPU model names (a100, l40s, h100), compute capability levels (sm_75, sm_80), and minimum VRAM (vram40, vram80). Combine them with &: --constraint="a100&vram80".

salloc — Reserving resources for multiple commands¶

srun --pty bash gives you a single interactive shell tied to one allocation. When you exit, the allocation is gone. If you need to run multiple commands against the same set of reserved resources (common in MPI development), use salloc instead:

salloc --ntasks=4 --mem=32G --time=04:00:00

For GPU work, salloc is also a natural fit:

salloc --gpus=1 --partition=gpu-preempt -q short --mem=16G --time=02:00:00

salloc reserves the resources and returns you to a shell on the login node, but with the allocation active. You then use srun to dispatch work to the reserved compute node(s):

# Resources are now reserved — run commands against them:
srun ./my_mpi_program
srun --ntasks=4 ./another_mpi_program

# Release the allocation when done:
exit

This is particularly useful for MPI development: you reserve four tasks once, then iterate quickly (edit code, recompile, re-run) without re-queuing for every test.

Common patterns¶

A few practical ways interactive jobs fit into a real research workflow:

Testing before batch submission. Get an interactive node, run your command with a small input, confirm it works and finishes in the expected time, then write the batch script with confidence. This avoids submitting batch jobs that fail in the first 30 seconds due to a missing module or wrong file path.

Debugging a failing batch job. When a batch job fails, request an interactive node with the same resource spec as the failing job, then reproduce the steps manually. You can inspect environment variables, check file permissions, and run the failing command directly.

Exploratory data analysis. Get a node with substantial RAM (--mem=64G or more) to load large datasets into Python or R interactively. This is far more practical than submitting and waiting for a batch job every time you want to try something.

Limitations¶

Interactive jobs are well-suited for development and debugging, but not for production runs:

• Disconnection ends the job. If you close your laptop, lose network connectivity, or your SSH session times out, the interactive job may be terminated. All progress in that session is lost.
• You must be present. You have to stay connected for the duration, which is impractical for long analyses.
• Time limits are enforced. Slurm will kill your session when the time limit is reached, whether or not you're in the middle of something.

For anything that runs longer than a few hours or that you don't need to supervise, use a batch script. See Submit Your First Job for the full batch workflow.