Chapter 5: AI software on LUMI (modules and containers)

When you download an AI project from the internet, the instructions usually say: "Just run pip install -r requirements.txt."

Why pip install is the "forbidden command"

LUMI uses a specialised high-performance storage system called Lustre. Lustre is designed for "Big Data" - it is incredibly good at reading and writing massive files (like gigabytes of model weights or terabytes of text) at lightning speeds. However, Lustre has a weakness: many small files.

A typical pip install of a library like PyTorch creates tens of thousands of tiny files. If everyone ran it, the filesystem would struggle to keep track of millions of tiny files, slowing down the entire supercomputer for everyone.

The solution: Containers

To avoid the "Million File" problem, we use Containers. On LUMI, our Container tool of choice is called Singularity (nowadays also known under its new name, Apptainer).

What is a Container?

Think of a shipping container. Instead of loading thousands of loose items onto a ship, you pack everything - your software, your specific version of Python, and your AI libraries - into one large, sealed "box."

On LUMI, this "box" is a single file (usually ending in .sif).

• For Lustre: Instead of tracking 20,000 tiny files, it only has to track one big Container file. This keeps the system fast.
• For You: Your software is "frozen" inside that box. This means it will work exactly the same way every time, regardless of what updates happen to the rest of the supercomputer.

How to get your AI software

You don't necessarily need to learn how to build these Containers from scratch. The LUMI AI Factory provides them for you at /appl/local/laifs/containers/. Most of example scripts and guides already contain code necessary to use a Container and you don't need to choose one yourself.

Below are the three main ways to interact with a Container. If you would like to test the first two options out right now, you can replace <container.sif> in the examples below with our default AI container: /appl/local/laifs/containers/lumi-multitorch-latest.sif.

1. Interactive Shell: You can "enter" the Container to get a live Command Line inside it. All commands you type here will be executed from "inside" the Container and will use the libraries packed inside it:

   user@LUMI: ~

   Copy−▢×

   user@LUMI:~$ singularity shell <container.sif>

   Once you "enter" a Container, you will see that the Command Line has changed. You can check what libraries are included in the Container by running pip list:

   user@LUMI: ~

   Copy−▢×

   user@LUMI:~$ pip list

   (Type exit or press Ctrl+D to leave the container when you are done).

2. Execute a single command: The second way to execute a command in the Container is with singularity exec, which enters the Container, executes the command and immediately exits the Container. For example, to list the libraries without staying inside:

   user@LUMI: ~

   Copy−▢×

   user@LUMI:~$ singularity exec <container.sif> pip list

3. Run a script: The most common way you will use Containers in your actual Jobs is by "running" them. You don't need to test this command right now, but you will use it later.

   user@LUMI: ~

   Copy−▢×

   user@LUMI:~$ singularity run <container.sif> python my_script.py

   This relies on the Container having a built-in "Runscript" - a default set of instructions baked in by the creator of the Container. You can optionally add commands at the end (like python my_script.py) to tell the Container exactly what script to execute.

Read more about Containers and interacting with them. (Optional)

Modules

Besides Containers, we use something called "Modules". These are software packages already installed by the LUMI staff. You "load" them into your session with a simple command, similar to turning on a light switch:

user@LUMI:~$ module purgeuser@LUMI:~$ module use /appl/local/laifs/modulesuser@LUMI:~$ module load lumi-aif-singularity-bindings

• module purge - Clears any previously loaded Modules, giving you a clean slate. This prevents conflicts between incompatible software.
• module use /appl/local/laifs/modules - Tells LUMI where to look for the AI Factory's Modules. By default, LUMI only knows about its own system Modules; this line adds our collection to the list.
• module load lumi-aif-singularity-bindings - Activates a specific Module. In this case, it sets up the necessary "Bindings" that allow your Container to communicate with LUMI's hardware (GPUs, high-speed network, etc.).

Summary checklist

• You understand that Lustre is great for big files, but struggles with many small files.
• You understand that pip install can slow down the system for everyone by creating too much "metadata."
• You understand that Singularity is the secure, supercomputer-friendly alternative to Docker that turns thousands of files into one easy-to-manage .sif file.

Knowledge check