Have you ever wondered how to run stuff on Stanford’s Sherlock Cluster but don’t know how and don’t want to read the manual? If so this post is for you!

Background

Singularity is an open source containerization software for running programs in reproducible environments.

You could run programs on the cluster’s bare metal, but this has a few drawbacks:

1. The software versions on the cluster are ancient, so the new features will not be available.
2. The setup is more hardware dependent, making your work less reproducible
3. Installing applications is not as straightforward as invoking a package manager.

Install signularity on your local machine. This is the only software you need to install.

Building

Drop these files in the same directory:

minimal.def:

Bootstrap: docker
From: nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04

%labels
	MAINTAINER Leni Aniva <v@leni.sh>

%files
	./diag.sh /bin/diag.sh

%runscript
	exec /bin/diag.sh

diag.sh:

#!/usr/bin/env sh
# diag.sh

nvidia-smi > $PWD/smi.txt

Then run the command as root.

sudo singularity build minimal.simg minimal.def

This builds a singularity image.

Copy the singularity image along with run.sh, given below, onto the cluster:

#!/bin/bash
#
#SBATCH --job-name=minimal
#
#SBATCH --time=10:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=2G
#SBATCH -C "GPU_GEN:PSC|GPU_GEN:VLT"

singularity run --nv ./minimal.simg

and run

sbatch run.sh

This should generate a smi.txt file containing the output of nvidia-smi, showing you that GPU is available.

Explanation

Singularity operates on standardized container formats that are compatible with dockers. This is the reason why it can directly pull from DockerHub. The reason Sherlock uses Singularity and not Docker or Nix is that singularity does not require the user to run containers as root, which would present a security risk.