Dartmouth Research Computing & Data (RCD) logo
  • Home 
  • HPC 
  • AI 
  • Services 
  • Status 
  • About Us 

  •   Search this site
  •  
  1.   HPC
  1. Home
  2. HPC
  3. Create a container image for HPC

Create a container image for HPC

On this page
Overview   Requirements   Create a Python file with demo code   Run the Python file to test it   Create a Dockerfile   Build the image locally   Run a container locally   Export the image   Transfer the image to Discovery   Build an Apptainer image from the Docker image   Run an Apptainer container   Create Slurm job script   Submit the job to Discovery  

Overview  

This KB outlines how to do the following:

  • Create a Docker image on your laptop
  • Export the image as a file and transfer it to a HPC server
  • Convert the image to an Apptainer (formerly Singularity) image
  • Run a container

Requirements  

  • An HPC/DartfS account (request at https://rcweb.dartmouth.edu/accounts/)
  • Install Rancher Desktop or Docker Desktop. You can check Docker Desktop license information at https://www.docker.com/pricing/faq/ where they list that use is free for research at non-for-profit institutions (last checked 2023-07-26).

Create a Python file with demo code  

cat << 'EOF' > fibonacci.py
import os, pickle, sys, time

CHECKPOINT_FILE = "fibonacci.pickle"

class Fibonacci:

  def __init__(self, n):
    self.checkpoint_i = 1
    self.n = n
    self.a = 0
    self.b = 1

  def run(self):
    print(f"Fibonacci: {self.n} (starting at calc {self.checkpoint_i})")
    self.checkpoint_time = time.time()
    if self.n == 0:
      return 0
    elif self.n == 1:
      return self.b
    else:
      for i in range(self.checkpoint_i, self.n):
        self.c = self.a + self.b
        self.a = self.b
        self.b = self.c
        if time.time() - self.checkpoint_time > 10:
          print(f"Checkpoint: {i}")
          self.checkpoint_i = i + 1
          with open(CHECKPOINT_FILE, 'wb') as f:
            pickle.dump(self, f)
          self.checkpoint_time = time.time()
      return self.b

# Handle calling Fibonacci from a script
if __name__ == "__main__":
  # allow for working with large numbers
  sys.set_int_max_str_digits(0)
  if os.path.exists(CHECKPOINT_FILE):
    with open(CHECKPOINT_FILE, 'rb') as f:
      fib = pickle.load(f)
  else:
    fib = Fibonacci(int(sys.argv[1]))
  # start/continue calculating
  result = fib.run()
  print(f"Result: {result}")
  # clean-up checkpoint file if exists
  if os.path.exists(CHECKPOINT_FILE):
    os.remove(CHECKPOINT_FILE)
EOF

Run the Python file to test it  

python ./fibonacci.py 30

Create a Dockerfile  

cat << 'EOF' > Dockerfile
FROM python:3.10-alpine

COPY fibonacci.py /src/
EOF

Build the image locally  

If not on a Mac with a M1 chip or later

docker build -t fibonacci:latest .

If on a Mac with a M1 chip or later, build the image for x86_64 architecture

docker buildx build --platform linux/amd64 -t fibonacci:latest .

Run a container locally  

docker run -it --rm fibonacci:latest python /src/fibonacci.py 30

Export the image  

docker save fibonacci:latest -o fibonacci.tar

Transfer the image to Discovery  

scp fibonacci.tar netid@discovery:

Build an Apptainer image from the Docker image  

ssh netid@discovery

apptainer build fibonacci docker-archive://fibonacci.tar

Run an Apptainer container  

apptainer exec fibonacci python /src/fibonacci.py 40

Create Slurm job script  

cat << 'EOF' > fibonacci-apptainer.sh
#!/bin/bash
#SBATCH --job-name=fibonacci-apptainer
#SBATCH --error=%x-%J.err
#SBATCH --output=%x-%J.out
#SBATCH --requeue
#SBATCH --time=00:01:00
#SBATCH --mem=1G
##SBATCH --account=free
##SBATCH --partition=standard

scontrol show job $SLURM_JOBID > fibonacci-info.$SLURM_JOBID

stdbuf -i0 -o0 -e0 apptainer exec fibonacci.tar.sif python -u /src/fibonacci.py 3000001
EOF

Submit the job to Discovery  

sbatch fibonacci-apptainer.sh

NOTE: You can use the squeue command to see the status of your job.

 Connect To A Linux Server with SSH (Secure Shell)
DartFS and High Performance Computing - Connecting to Linux machines via command line / terminal 
On this page:
Overview   Requirements   Create a Python file with demo code   Run the Python file to test it   Create a Dockerfile   Build the image locally   Run a container locally   Export the image   Transfer the image to Discovery   Build an Apptainer image from the Docker image   Run an Apptainer container   Create Slurm job script   Submit the job to Discovery  
Copyright © 2025 Dartmouth Research Computing & Data | Powered by Hinode.
Dartmouth Research Computing & Data (RCD)
Code copied to clipboard