Container Technology

Building more complex processing and analysis pipelines within a container can offer several advantages:

  • The container fully encapsulates the job and all software dependencies
  • Software dependencies do not need to be installed on the HPC facility
  • Your workflow can easily be replicated or moved between other systems
  • You can run pre-built job containers which application developers publish
  • You can build your own containers to move between systems, or share with your group

The Comet HPC facility supports the following container tools:

Unlike Rocket, Comet is design to be fully native container aware - all tools are configured to allow the running of container jobs under Slurm from the above technologies, enabling the community to take advantage of the increased number of ready-built container images available to them from various software developers.

Please note that we do not have support for Kubernetes at this time.

This section is incomplete

This documentation section on Container Technology is still being written and will not be complete until the Comet HPC facility is fully commissioned.

Docker

Apptainer

If you have used Singularity previously, then Apptainer is a 100%, like-for-like replacement. You can create container images which exist in a single file and are easy to transport between different systems and provide a consistent environment for your software, reducing the need for the installation of additional libraries and other dependencies.

You can use Apptainer to build and run container images on the Comet login nodes, as well as run container images on any of the available compute nodes.

Please read our full guide on using Apptainer on Comet, which is linked below, or can be accessed from the Advanced Software Topics section of the wiki:

Singularity

Singularity is now deprecated and the community have largely moved to Apptainer instead - an alternative version of Singularity now exists named SingularityCE; we do not support SingularityCE.

Singularity CE

SingularityCE is the 'free' version maintained by a commercial entity (Sylabs). This software is not installed on Comet and we do not support it.

Apptainer

Apptainer is a like-for-like, drop-in replacement for Singularity, is open-source and supported by the Linux foundation. We have this tool installed on Comet and we provide support for it.

If you have previously used Singularity, then you can start using Apptainer by simply changing any call to singularity to apptainer instead. Historically we did have Singularity on our previous Rocket HPC facility, but it was not advertised and was very much a “use at your own risk” capability.

You are encouraged to read our Apptainer help guide to be aware of any specification configuration or limitations with the use of the technology on Comet.

Podman

Podman is a container technology tool somewhat similar to Docker, but has a slightly different implementation making it easier to integrate into existing systems; it is often preferred to Docker due to being easier to run by a non-admin user. From a user perspective it has much of the same rich functionality as Docker and is more suited to expert users than something like Apptainer.

Using Podman to build or run containers on the Comet login nodes as well as run containers on the Comet compute nodes is fully supported.

Please read our Podman help guide for a walk-through of how to use Podman on Comet.

Back to Advanced Topics