Nextflow

Nextflow is a domain specific language and service which enabled repeatable, reproducible compute environments. It is intended for researchers and scientists to leverage existing, complex data analysis pipelines without the need to re-implement or create those environments themselves.

This involves:

• A job service which runs on (one or more) head node(s).
• Submission of scripts to the Nextflow service.
• The nextflow service packaging the script and pipeline and launching the job (via Slurm) on behalf of the user.

It has the following requirements:

• Runs on Java
• Needs Singularity or Apptainer for container management
• Service running on head nodes - only accessible by users on the head nodes - not externally to HPC

We have the following mandatory requirements for Nextflow to work with Comet:

• Must be configured to submit based on user account code - otherwise no access to compute resources
• Jobs must run with the name of the user submitting the Nextflow request - no system users

Back to software