|
Why Manage My Own Software?
|
Software managed by other parties can be subject to change and difficult to reproduce in the future.
Research workflows require increasingly sophisticated software workflows to support them.
A small time investment up front in developing a robust software workflow can be invaluable in a research environment.
|
|
Separating Packages Using Environment Modules
|
A software environment is defined by a set of environment variables and the directories they refer to.
Environment modules are tcl (or lua) scripts specialized for modifying software environments.
The module command lets you quickly swap between software environments.
|
|
Managing Software Installed via Make
|
The configure; make; make install workflow is one of the most common installation methods for software.
Most of the customizations to the workflow happen during the configure step.
Saving the source code and install recipes for your packages and environments will make your life easier.
|
|
Installing Modules Using CMake
|
CMake is a more modern and advanced build system that is popular for packages with complex sets of dependencies.
We can still customize it to our needs for the same modulefile workflow we’ve been using.
|
|
Working with Anaconda and Pip
|
Conda environments are complex and finicky, but we can avoid most issues by forcing them to install into our module framework.
We can use conda to isolate pip installations.
|
|
Apptainer: Getting started
|
apptainer is another container platform and it is often used in cluster/HPC/research environments.
apptainer has a different security model to other container platforms, one of the key reasons that it is well suited to HPC and cluster environments.
apptainer has its own container image format (SIF).
The apptainer command can be used to pull images from apptainer Hub and run a container from an image file.
|
|
Apptainer: Practical Considerations
|
Apptainer caches downloaded images so that an unchanged image isn’t downloaded again when it is requested using the apptainer pull command.
You can free up space in the cache by removing all locally cached images or by specifying individual images to remove.
Your current directory and home directory are usually available by default in a container.
You have the same username and permissions in a container as on the host system.
You can specify additional host system directories to be available in the container.
Apptainer can start a container from a Docker image which can be pulled directly from Docker Hub.
|