Environment Modules¶

On your laptop, installing software is straightforward: You download it, install it, and it's available everywhere. HPC clusters work differently. Hundreds of users share the same system, often needing different (and incompatible) versions of the same software. Environment modules are how clusters solve this problem.

The Problem Modules Solve¶

Imagine a cluster where every piece of software is installed globally, the way you'd install apps on your laptop. One research group needs Python 3.9 for a legacy pipeline. Another needs Python 3.12 for a new project. A third group needs a version of NumPy compiled against a specific version of MKL. A bioinformatics lab needs GCC 10 to build their tools, while a physics group needs GCC 13 for C++20 support.

Installing all of these globally creates conflicts. Libraries compiled against one version of a compiler may crash when loaded alongside another. Two versions of the same tool can't both live at /usr/bin/python3. And if a sysadmin upgrades a shared library to satisfy one group, it may silently break another group's workflow.

HPC systems are shared infrastructure. Unlike your laptop, you don't get to decide what's installed system-wide. Environment modules solve this by letting each user choose exactly the software they need, in the versions they need, without affecting anyone else on the system.

What Modules Actually Do¶

If you've read Advanced Linux: Permissions, Pipes & the Environment, you know that your shell uses environment variables like $PATH to find commands. When you type python3, your shell searches through the directories listed in $PATH until it finds a matching executable.

Modules work by manipulating these environment variables. When you load a module, it prepends the software's bin/ directory to your $PATH, adds its lib/ directory to $LD_LIBRARY_PATH, and sets any other variables the software needs. When you unload it, those changes are reversed.

That's all! There's no containerization or virtualization, just careful management of environment variables.

Core Commands¶

These are the commands you'll use every day on Unity. They all start with module.

See what's available¶

module avail

This lists every module installed on the cluster. The output can be overwhelming because Unity has many modules. To narrow it down, pass a search term:

module avail gcc

This shows only modules whose names contain "gcc." Use this when you know roughly what you need but aren't sure of the exact version or name.

Load a module¶

module load gcc/12.2.0

This makes GCC 12.2.0 available in your current session. After running this, gcc points to version 12.2.0 and any associated tools (like g++) are also on your $PATH.

You can load multiple modules at once:

module load gcc/12.2.0 openmpi/4.1.4

See what's loaded¶

module list

This shows every module currently active in your session. If something isn't behaving as expected, this is the first thing to check; the software you need might not be loaded.

Unload a module¶

module unload gcc/12.2.0

This reverses the environment changes that module load made. The software is no longer available in your session.

Start fresh¶

module purge

This unloads everything. It's useful when you've accumulated modules over a session and want a clean slate. Many experienced users start their job scripts with module purge to ensure a predictable environment.

Inspect a module¶

module show gcc/12.2.0

This reveals exactly what a module does: which directories it adds to $PATH, what environment variables it sets, and whether it loads any other modules as dependencies. This is invaluable for debugging:

prepend-path    PATH            /opt/software/gcc/12.2.0/bin
prepend-path    LD_LIBRARY_PATH /opt/software/gcc/12.2.0/lib
setenv          CC              /opt/software/gcc/12.2.0/bin/gcc

Understanding Module Names¶

Modules follow a name/version naming convention:

gcc/12.2.0
cuda/12.1
openmpi/4.1.4
uv/0.1.24

The name identifies the software, and the version after the slash identifies the specific release. Some key things to know:

• Default versions: If you run module load gcc without specifying a version, the system loads a default (usually marked with (D) in module avail output). It's better to always specify the version explicitly. The default may change when the system is updated, which could break your workflow.

• Partial matching: You can often use partial version numbers. module load gcc/12 may resolve to gcc/12.2.0 if that's the only 12.x installed.

• Hierarchical modules: Some modules only appear after you've loaded a prerequisite. For example, MPI libraries compiled with a specific compiler may only show up in module avail after you load that compiler. If you can't find a module you expect to exist, check whether it depends on another module being loaded first.

Loading Modules in Job Scripts¶

This is the single most important thing to understand about modules on a cluster: modules loaded in your interactive session do not carry over into your batch jobs.

When you submit a job with sbatch, it starts a fresh shell on a compute node. That shell has no memory of what you loaded on the login node. If your job needs a specific tool like uv, you must load it in the job script.

#!/bin/bash
#SBATCH --job-name=analysis
#SBATCH --ntasks=1
#SBATCH --mem=4G
#SBATCH --time=01:00:00

module purge                  # start clean
module load uv                # load what you need

uv run python my_analysis.py

The module purge at the top ensures you're starting from a known state, regardless of what might be loaded by default. Then you explicitly load exactly what your job requires. This makes your job script self-contained and reproducible: anyone can read it and know exactly what software environment it expects.

For more on job script structure, see Submit Your First Job.

Common Pitfalls¶

When NOT to Use Modules¶

Modules are the right tool for system-level software: compilers (GCC, Intel), MPI implementations (OpenMPI, MVAPICH), GPU toolkits (CUDA), and core scientific libraries. These are complex to build, need to be compiled for the specific cluster hardware, and are shared across many users.

Modules are not the right tool for managing Python packages, R libraries, or other language-specific dependencies. For those, use the language's own package manager inside an isolated environment:

• Python packages: Use virtual environments managed by a tool like uv. See Getting Started with uv for a complete walkthrough. Instead of loading a Python module, you will module load uv and let uv download and manage the exact Python version your project needs.

• R packages: Install them into a user library with install.packages() inside an R session.

The distinction is about scope. Modules manage software that's shared across the cluster. Virtual environments and user libraries manage dependencies that are specific to your project. Mixing these up (trying to use modules for everything or ignoring modules entirely) leads to environments that are brittle and hard to reproduce.

Quick Reference¶

What's Next¶

• Submit Your First Job: Put your module knowledge to work in a real batch job
• Getting Started with uv: Set up isolated Python environments on the cluster
• MPI Hello World: Load compiler and MPI modules to run your first parallel program