Running Containers with Apptainer¶

New to containers? Read Containers on HPC: An Introduction first for the conceptual background and a decision framework for when to use them.

Load Apptainer¶

Apptainer is available on Unity as a module. Load it before running any apptainer commands:

module load apptainer/latest

Add this line to your batch scripts too, so the commands are available when the job runs.

Step 1: Pull an image¶

Apptainer can pull images directly from Docker Hub or any Docker-compatible registry. Pulled images are converted to Apptainer's native .sif format, a single portable file.

# Pull Ubuntu 22.04 from Docker Hub
apptainer pull docker://ubuntu:22.04

# Pull a GPU-optimized PyTorch image from NVIDIA's registry (NGC)
apptainer pull docker://nvcr.io/nvidia/pytorch:24.01-py3

Each command produces a .sif file in the current directory (e.g., ubuntu_22.04.sif, pytorch_24.01-py3.sif).

cd /path/to/your/scratch
apptainer pull docker://nvcr.io/nvidia/pytorch:24.01-py3

Pulling a large image takes time and network I/O, so do it once on a login node and reuse the .sif file in subsequent jobs.

Step 2: Run Commands in the Container¶

There are three main ways to interact with a container:

apptainer exec — run a specific command¶

apptainer exec ubuntu_22.04.sif cat /etc/os-release

This launches the container, runs cat /etc/os-release inside it, prints the output, then exits. The container process is gone when the command finishes. Use exec when you want to run your script or tool as part of a larger workflow.

# Run a Python script using the container's Python
apptainer exec pytorch_24.01-py3.sif python train.py --epochs 50

apptainer run — run the image's default entry point¶

Some images define a default command (their entry point). apptainer run executes that default. It is equivalent to docker run without specifying a command. Many scientific images don't define a meaningful default, so exec is more generally useful.

apptainer run ubuntu_22.04.sif

apptainer shell — interactive shell inside the container¶

apptainer shell ubuntu_22.04.sif

This drops you into a shell inside the container. Useful for exploring the environment, debugging, or prototyping. Your shell prompt changes to Apptainer> to remind you where you are.

Apptainer> python3 --version
Python 3.10.12
Apptainer> exit

Type exit or press Ctrl+D to leave the container shell and return to your normal cluster session.

Step 3: Bind Mounts — Accessing Your Files¶

By default, the container has its own isolated filesystem. Your home directory, scratch space, and project directories are not visible unless you mount them explicitly.

Use --bind (or -B) to mount paths from the cluster into the container:

# Mount a scratch workspace as /data inside the container
apptainer exec --bind /path/to/your/scratch:/data \
    pytorch_24.01-py3.sif python /data/train.py

The syntax is --bind /host/path:/container/path. You can mount multiple paths:

apptainer exec \
    --bind /path/to/your/scratch:/data \
    --bind /path/to/my/code:/code \
    pytorch_24.01-py3.sif python /code/train.py --data-dir /data

Persistent bind paths with $APPTAINER_BINDPATH

If you always bind the same directories, set this environment variable in your ~/.bashrc or at the top of your job script:

export APPTAINER_BINDPATH="/path/to/your/scratch:/data"

Apptainer reads this variable and applies those mounts to every container you run, without needing --bind each time.

Step 4: GPU Passthrough¶

Add the --nv flag to expose the host's NVIDIA drivers and CUDA libraries to the container:

apptainer exec --nv pytorch_24.01-py3.sif python -c "
import torch
print(torch.cuda.is_available())
print(torch.cuda.get_device_name(0))
"

The --nv flag injects the host driver into the container at runtime. The container image itself does not need to include the CUDA driver, only the CUDA toolkit libraries that match the host driver version.

You must request GPU resources from Slurm to actually have a GPU available when the job runs (see the batch job example below).

Step 5: Building Custom Images¶

When an existing image doesn't quite fit your needs, you can build your own. Apptainer uses definition files (.def) to describe how to build an image.

Here's a minimal example that starts from an existing Docker image and adds Python packages on top:

Bootstrap: docker
From: ubuntu:22.04

%post
    apt-get update -y
    apt-get install -y python3 python3-pip
    pip3 install numpy scipy matplotlib

%environment
    export LC_ALL=C
    export PATH=/usr/local/bin:$PATH

%labels
    Author mynetid
    Version 1.0

%help
    A minimal Ubuntu 22.04 image with NumPy, SciPy, and Matplotlib.

Save this as myenv.def, then build the image:

apptainer build --ignore-fakeroot-command myenv.sif myenv.def

The --ignore-fakeroot-command flag tells Apptainer to skip the fakeroot helper even if it is not configured on the system. On Unity, this is the recommended way to build images without root access.

# On your local machine
apptainer build myenv.sif myenv.def

# Copy to cluster scratch
scp myenv.sif login.unityhpc.org:/path/to/your/scratch/

Step 6: Running Containers in a Slurm Job¶

apptainer exec slots naturally into a batch script. Just replace the bare Python call with the containerized version:

#!/bin/bash
#SBATCH --job-name=container-train
#SBATCH --partition=gpu
#SBATCH --time=04:00:00
#SBATCH --cpus-per-task=4
#SBATCH --mem=32G
#SBATCH --output=%x_%j.out
#SBATCH --error=%x_%j.err
#SBATCH --gres=gpu:1

# Run your work
# Path to your SIF file (store in a scratch workspace, not home)
SIF=/path/to/your/scratch/pytorch_24.01-py3.sif

# Bind scratch into the container at /data
apptainer exec --nv \\
    --bind /path/to/your/scratch:/data \\
    "$SIF" \\
    python /data/train.py --output /data/results

Key points for container jobs:

• Set SIF as a variable at the top for easy swapping without hunting through the script.
• Use --nv if your workload uses the GPU.
• Use --bind to expose your data and output directories.
• No module load is needed for the software inside the container; it's all bundled in the image. You may still need to load cluster modules for things outside the container (e.g., if launching MPI across nodes).

Common Pitfalls¶

apptainer exec --bind /my/path:/mnt myimage.sif ls /mnt

apptainer exec --writable-tmpfs myimage.sif my_tool