====== CASTEP ====== > CASTEP is a leading code for calculating the properties of materials from first principles. Using density functional theory, it can simulate a wide range of properties of materials proprieties including energetics, structure at the atomic level, vibrational properties, electronic response properties etc. In particular it has a wide range of spectroscopic features that link directly to experiment, such as infra-red and Raman spectroscopies, NMR, and core level spectra. * For more information: https://www.castep.org/ * Expanded user documentation for CASTEP is available on the community documentation website: https://castep-docs.github.io/castep-docs/ We have the standard (CPU) version of **CASTEP** (in //serial// and //MPI// builds) available on Comet. * CASTEP CPU - serial / MPI : Version 25.12 Whilst there is a version of CASTEP with GPU acceleration, this was considered a //beta// release of an older version (23.1), has not been updated, and is restricted to older versions of the Nvidia HPC / CUDA SDK. As such, we have not installed it on Comet. The commercial [[https://www.3ds.com/products/biovia/materials-studio|BIOVIA Material Studio]] which incorporates CASTEP //does// have GPU support, but is not currently licensed for use on Comet. ---- ===== Running CASTEP on Comet (CASTEP CPU) ===== CASTEP on Comet is provided as an [[advanced:apptainer|Apptainer container environment]] - please read the Apptainer guide //first//. The CASTEP container images are stored in the following location: * ''/nobackup/shared/containers/castep-25.12.sif'' - **CPU** (serial & MPI) version You will also find a matching shell script: * ''/nobackup/shared/containers/castep-25.12.sh'' To run CASTEP, simply ''source'' the shell script: $ source /nobackup/shared/containers/castep-25.12.sh You can then use a helper command ''container.run'' to run commands inside the container, without needing pass huge numbers of parameters to the ''apptainer exec'' command - this automatically adds ''bind'' arguments to let you access ''/scratch'' and ''/nobackup'' inside the container: $ source /nobackup/shared/containers/castep-25.12.sh $ container.run castep.serial -h You only need to ''source'' the shell script once, either at the start of your Comet session, if working interactively, or at the top of your sbatch file if submitting a job. All of the CASTEP binaries and scripts are available under ''/opt/castep'' inside the container: $ apptainer exec /nobackup/shared/containers/castep-25.12.sif ls /opt/castep/bin MolPDOS castep2cube castepclean cif2cell den2xsf md2xyz pdb2xtl vasp2pdb __pycache__ castep2force.py castepconv.py cssr2cell dispersion.pl md_to_phonons.py perl_md.pl xtl2cell awk_elf_to_xsf castep2ngwst cell2cell cssr2gulp dos.pl mdtep perl_pimd.pl xtl2cssr bands2dos castep2pdb cell2pdb cube_add elastics.py mode_follow phonon2xyz xtl2pdb bs_sc2pc castep2shak cell2sgroup cube_info f2py numpy-config phonon_kpoints xtl2sgroup c2x castep2shx cell2shx cube_subtract generate_strain.py od2od phonons xtl2shx castep.mpi castep2w90 cell2xsf cube_to_xsf geom2dcd optados pimerge.pl castep.serial castep2xsf cell2xtl current2nics geom2pdbseq orbitals2bands pot1d castep2casino castep2xtl cell2xyz den2cube geom2xsf pdb2cell shx2cell castep2cell castep2xyz celltools den2grd geom2xtl pdb2pdb symmetry_snap castep2cif castep_GA ceteprouts.pm den2vasp geom2xyz pdb2shak vasp2cell castep2cssr castep_md_reader.py charge2d den2xplor geom2xyz.pl pdb2shx vasp2cif $ Within the container, the directory ''/opt/castep/bin'' is added to the ''PATH'', so you do __not__ need to prefix it to any commands - see the non-MPI and MPI examples below. === CASTEP CPU - Serial/Non-MPI === Now you can call the ''castep.serial'' binary inside the container. A simple example to show the CASTEP help page is shown below: $ source /nobackup/shared/containers/castep-25.12.sh $ container.run castep.serial -h Usage: castep : Run files .cell [and .param] " [-d|--dryrun] : Perform a dryrun calculation on files .cell " [-s|--search] : print list of keywords with match in description " [-v|--version] : print version information " [-h|--help] : describe specific keyword in <>.cell or <>.param " " all : print list of all keywords " " basic : print list of basic-level keywords " " inter : print list of intermediate-level keywords " " expert : print list of expert-level keywords " " dummy : print list of dummy keywords $ === CASTEP CPU - MPI === Call the ''castep.mpi'' binary: $ source /nobackup/shared/containers/castep-25.12.sh $ container.run castep.mpi -h Usage: castep : Run files .cell [and .param] " [-d|--dryrun] : Perform a dryrun calculation on files .cell " [-s|--search] : print list of keywords with match in description " [-v|--version] : print version information " [-h|--help] : describe specific keyword in <>.cell or <>.param " " all : print list of all keywords " " basic : print list of basic-level keywords " " inter : print list of intermediate-level keywords " " expert : print list of expert-level keywords " " dummy : print list of dummy keywords $ ---- ==== Accessing Data ==== This guide is no longer relevant if you ''source'' the shell script as detailed above - all of these steps will be performed automatically for you. By default a container running under Apptainer automatically has access to your home directory - i.e. if ''castep'' tries to read or write files in ''$HOME'', it will be mapped to your real home directory on Comet. However the ''/nobackup'' directory is not included by default. If you want ''castep'' to read or write data to or from your project directory, prefix the ''apptainer exec'' call as follows: $ apptainer exec --bind /scratch:/scratch --bind /nobackup:/nobackup /nobackup/shared/containers/castep-25.12.sif castep.serial With this //bind// option, ''castep'' can read and write all of your normal group folders and files under the ''/nobackup'' directory, in addition to anything in your ''$HOME''. The //example// **run** file linked above shows an example of the use of these ''bind'' options. ---- ===== Limitations ===== Please note that due to limitations in compatibility between the CASTEP source code and more modern releases of the Python build tools (e.g. ''meson'' and similar), it has **not** been possible to install the CASTEP to Python API integration features (e.g. from ''make castep_python''). ---- ===== Building CASTEP for Comet ===== **Important!** The following sections are //only// relevant to Comet RSE staff or those needing to build CASTEP from source themselves. //Users// of CASTEP should //ignore// this section. === Apptainer container definition (CPU version) === Build requirements * ''autoconf'' * ''build-essential'' * ''cmake'' * ''f90wrap'' - Python module * FFTW3 - Uses [[https://github.com/amd/amd-fftw|AMD provided]] re-implementation of FFTW3, compiled from source, optimised for AMD Zen 5 architecture * ''gcc-14'' and ''gfortran-14'' * ''git'' * ''gzip'' * ''libsymspg-dev'' * ''meson'' * ''numpy'' - Python module * ''libgomp1'' * ''openmpi-bin'', ''openmpi-common'' and ''libopenmpi-dev'' * LAPACK - Compiled from source and optimised for AMD Zen 5 architecture * ''mpi4py'' - Python module * OpenBLAS - Compiled from source and optimised for AMD Zen 5 architecture * ''PyCifRW'' - Python module * ''python3'' and ''python3-pip'' * ''ply'' - Python module * ''scipy'' - Python module * ''tar'' * ''wget'' Some of these need to be installed manually as part of the base OS or container environment, a small number of the Python modules are automatically installed by ''pip'' which is called during the ''make install'' phase of the compilation of CASTEP. Some additional Python modules are //not// installed automatically by CASTEP and need to be added prior to the //make check// or //make install// phase. Compiler settings for anything built from source in this CASTEP environment use the settings below, ensuring optimal execution on the AMD Epyc architecture. Note that the ''OPENBLAS_TARGET'' variable is the [[https://github.com/OpenMathLib/OpenBLAS|recommended setting]] for OpenBLAS on AMD Zen 4 (and higher) processors - it is __not__ a mistake: export CFLAGS="-O3 -march=znver5 -pipe" export OPENBLAS_TARGET=SKYLAKEX This simple script, ''castep_cpu_build.sh'', maps the current directory (which should contain ''CASTEP-25.12.tar.gz'' - this is **not** provided freely due to licensing terms) and calls the ''apptainer build'' command using the definition file, ''castep_cpu_build.def'', provided below. **Build script:** #!/bin/bash echo "Loading modules..." module load apptainer echo "OK" echo "" echo "Building container..." export APPTAINER_TMPDIR=/scratch # You **must** provide the CASTEP-25.12.tar.gz file in a directory # and then specify it in the bind parameter to apptainer as below. # By default we try to find it in the same directory as this script. CASTEP_TAR_FOLDER=`pwd` if [ -s "$CASTEP_TAR_FOLDER/CASTEP-25.12.tar.gz" ] then echo "- Found: $CASTEP_TAR_FOLDER/CASTEP-25.12.tar.gz" else echo "- ERROR: $CASTEP_TAR_FOLDER/CASTEP-25.12.tar.gz not found" exit 1 fi apptainer build --bind=$CASTEP_TAR_FOLDER:/mnt castep.sif castep_cpu_build.def 2>&1 | tee castep.log **Container definition file:** Bootstrap: docker From: ubuntu:noble #################################################################### # # CASTEP 25.12 # # NOTE 1: This build is for the CPU-only version of CASTEP. # # NOTE 2: This build relies on CASTEP-25.12.tar.gz being mounted # in a folder at /mnt/ at the time this build file is run. Check # castep_cpu_build.sh to see how this is accomplished. # # NOTE 3: The CASTEP source code, whilst free, is *not* freely # distributed. See https://www.castep.org/get_castep # #################################################################### %post # Prevent interactive prompts export DEBIAN_FRONTEND=noninteractive #################################################################### # # Basic system packages # #################################################################### # Update & install only necessary packages apt-get update apt-get install -y apt-utils wget autoconf cmake build-essential tar git aptitude python3-pip gcc-14 gfortran-14 openmpi-bin openmpi-common libopenmpi-dev libgomp1 autoconf libsymspg-dev meson vim ln -s /usr/bin/python3 /usr/bin/python # Clean up APT cache to save space apt-get clean # Dependency for some castep python scripts pip3 install f90wrap --break-system-packages pip3 install numpy --break-system-packages pip3 install scipy --break-system-packages # Remove any Python cache files after pip pip3 cache purge ##################################################################### # # This is all the custom stuff needed to build CASTEP # ##################################################################### # This flag needs to be set to indicate which CPU architecture we # are optimising for. You cannot build for AMD on an Intel host, # as several of the 'make install' steps for FFTW3, OpenBLAS etc # attempt to execute code built during the install... and Intel # won't understand certain AMD opcodes and vice versa. AMD_ARCH=0 if [ "$AMD_ARCH" = "1" ] then # Compiling on AMD Epyc export BASE_CFLAGS="-O3 -march=znver5 -pipe" # On Zen5 arch, OpenBLAS suggests using Skylake-X support export OPENBLAS_TARGET=SKYLAKEX export MAKE_JOBS=8 else # Compiling on Intel Skylake+ - 12/13th gen core processors or higher - no AVX-512 support export BASE_CFLAGS="-O3 -march=skylake -pipe" # Target OpenBLAS to not use AVX-512 support export OPENBLAS_TARGET=HASWELL export MAKE_JOBS=8 fi export FFTW_CFLAGS="$CFLAGS -fopenmp" export CFLAGS="$BASE_CFLAGS" echo "" echo "Post-OS-install setup for CASTEP" echo "================================" # A download place for external libraries mkdir -p /src/zipped echo "" echo "1. Building optimised OpenBLAS" echo "==============================" cd /src/zipped/ && \ wget https://github.com/OpenMathLib/OpenBLAS/releases/download/v0.3.31/OpenBLAS-0.3.31.tar.gz && \ cd /src && \ tar -xzf zipped/OpenBLAS-0.3.31.tar.gz && \ cd /src/OpenBLAS-0.3.31 && \ CC=gcc-14 make -j$MAKE_JOBS TARGET=$OPENBLAS_TARGET && \ make install && \ ln -s /opt/OpenBLAS/lib/libopenblas.so /opt/OpenBLAS/lib/libblas.so && \ ln -s /opt/OpenBLAS/lib/libopenblas.so /opt/OpenBLAS/lib/liblapack.so && \ echo "/opt/OpenBLAS/lib" > /etc/ld.so.conf.d/openblas.conf if [ "$AMD_ARCH" = "1" ] then # This is for AMD arch only echo "" echo "2. Building AMD optimised FFTW3" echo "===============================" cd /src/zipped && \ wget https://github.com/amd/amd-fftw/archive/refs/tags/5.2.tar.gz && \ cd /src && \ tar -xzf zipped/5.2.tar.gz && \ cd /src/amd-fftw-5.2 && \ export OMPI_CC=gcc-14 && \ CC=gcc-14 CFLAGS=$FFTW3_CFLAGS AMD_ARCH=znver5 F77=gfortran-14 ./configure \ --enable-shared \ --enable-openmp \ --enable-mpi \ --enable-threads \ --enable-amd-opt \ --with-g77-wrappers \ --prefix=/opt/fftw3 && \ make -j$MAKE_JOBS && \ make install && \ echo "/opt/fftw3/lib" > /etc/ld.so.conf.d/fftw3.conf else # This is for non-AMD arch echo "" echo "2. Building FFTW3 (non-AMD)" echo "===========================" cd /src/zipped && \ wget https://www.fftw.org/fftw-3.3.10.tar.gz && \ cd /src && \ tar -zxf zipped/fftw-3.3.10.tar.gz && \ cd /src/fftw-3.3.10 export OMPI_CC=gcc-14 && \ CC=gcc-14 CFLAGS=$FFTW3_CFLAGS F77=gfortran-14 ./configure \ --enable-shared \ --enable-openmp \ --enable-mpi \ --enable-threads \ --with-g77-wrappers \ --prefix=/opt/fftw3 && \ make -j$MAKE_JOBS && \ make install && \ echo "/opt/fftw3/lib" > /etc/ld.so.conf.d/fftw3.conf fi echo "" echo "3. Building Lapack" echo "==================" cd /src/zipped && \ wget https://github.com/Reference-LAPACK/lapack/archive/refs/tags/v3.12.0.tar.gz && \ cd /src && \ tar -xzf zipped/v3.12.0.tar.gz && \ cd /src/lapack-3.12.0 && \ mkdir build && \ cd build && \ CC=gcc-14 FC=gfortran-14 cmake -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=/opt/lapack .. && \ CC=gcc-14 FC=gfortran-14 cmake --build . -j --target install && \ echo "/opt/lapack/lib" > /etc/ld.so.conf.d/lapack.conf && rm /etc/ld.so.cache && ldconfig echo "" echo "4. Building CASTEP" echo "==================" # Unpack the source tarball cd /src && \ tar -xzf /mnt/CASTEP-25.12.tar.gz # We have to do a little bit of symlinking here, as CASTEP assumes it is installing to /usr/local ... mkdir -p /opt/castep/bin mkdir -p /opt/castep/lib mkdir -p /opt/castep/local ln -s /opt/castep/bin /opt/castep/local/bin ln -s /opt/castep/lib /opt/castep/local/lib # Step in to the source tree cd /src/CASTEP-25.12 mkdir -p bin/linux_x86_64_gfortran10--serial mkdir -p bin/linux_x86_64_gfortran10--mpi # Edit the build rules to ensure we are # linking to the correct location for the installed LAPACK libs echo "" echo "4.1 Editing obj/platforms/linux_x86_64_gfortran10.mk" echo "================================================" cp -v obj/platforms/linux_x86_64_gfortran10.mk obj/platforms/linux_x86_64_gfortran10.mk.old && \ cat obj/platforms/linux_x86_64_gfortran10.mk.old | \ sed 's/MATH_LIBS = -llapack -lblas/MATH_LIBS = -L\/opt\/lapack\/lib -llapack -lblas/g' > obj/platforms/linux_x86_64_gfortran10.mk # Edit optimisation rules for AMD Zen 5 echo "" echo "4.2 Editing cmake/buildflags_gnu.cmake" echo "==================================" cp -v cmake/buildflags_gnu.cmake cmake/buildflags_gnu.cmake.old if [ "$AMD_ARCH" = "1" ] then cat cmake/buildflags_gnu.cmake.old | \ sed 's/march=mavx/march=znver5/g' | \ sed 's/march=native/march=znver5/g' > cmake/buildflags_gnu.cmake else cat cmake/buildflags_gnu.cmake.old | \ sed 's/march=mavx/march=skylake/g' | \ sed 's/march=native/march=skylake/g' > cmake/buildflags_gnu.cmake fi # Edit include path for Source/Utility/FreeIPC_c.c # This has an incorrect header include path "mpi.h" where it should have echo "" echo "4.3 Editing Source/Utility/FreeIPC_c.c" echo "======================================" cp -v Source/Utility/FreeIPC_c.c Source/Utility/FreeIPC_c.c.old cat Source/Utility/FreeIPC_c.c.old | sed 's/include "mpi.h"/include /g' > Source/Utility/FreeIPC_c.c echo "" echo "4.4 Now build/install CASTEP main binaries - SERIAL version" echo "===========================================================" # We call the compile and 'install' in one, to share the same flags - this is the serial version make -j$MAKE_JOBS install FFT_THREAD=true GRIMMED3=compile GRIMMED4=compile DL_MG=compile INSTALL_DIR=/opt/castep/bin CFLAGS="$BASE_CFLAGS" CC=gcc-14 BUILD=fast MATHLIBS=openblas MATHLIBDIR=/opt/OpenBLAS/lib FFT=fftw3 FFTLIBDIR=/opt/fftw3/lib echo "" echo "4.5 Now install CASTEP tools - SERIAL version" echo "=============================================" make install-tools echo "" echo "4.6 Now check CASTEP - SERIAL version" echo "=====================================" make check ################################################################################################## echo "" echo "4.7 Now build/install CASTEP main binaries - MPI version" echo "========================================================" # We call the compile and 'install' in one, to share the same flags - this is the MPI version # It has to be compiled independently of the serial version as several of the libs it creates # and links to are different in parallel mode. make -j$MAKE_JOBS FFT_THREAD=true GRIMMED3=compile GRIMMED4=compile DL_MG=compile INSTALL_DIR=/opt/castep/bin CFLAGS="$BASE_CFLAGS" CC=gcc-14 BUILD=fast COMMS_ARCH=mpi SUBARCH=mpi MATHLIBS=openblas MATHLIBDIR=/opt/OpenBLAS/lib FFT=fftw3 FFTLIBDIR=/opt/fftw3/lib cp -v obj/linux_x86_64_gfortran10--mpi/castep.mpi /opt/castep/bin ################################################################################################## ######################################### # # The make castep_python process does not work!!! # # Modern versions of Python (3.12+) build extensions with f2py using # 'meson' rather than 'distutils' ... unfortunately meson insists on # variables/variable names *not* having a '-' in them... and one of the # libraries that f2py wants to link with castep_python is libmctc-lib.a ... # which turns into -lmctc-lib, as well as the variable name mctc-lib # ... and meson promptly chokes on it. # # As documented here: https://github.com/mesonbuild/meson/issues/9754 # # Runing 'make castep_python' gets you an error like this: # ======================= # The Meson build system # Version: 1.3.2 # Source dir: /tmp/tmprbqjfprk # Build dir: /tmp/tmprbqjfprk/bbdir # Build type: native build # # meson.build:41:0: ERROR: Assignment target must be an id. # mctc-lib = declare_dependency(link_args : ['-lmctc-lib']) # ^ # A full log can be found at /tmp/tmprbqjfprk/bbdir/meson-logs/meson-log.txt # ======================= # # Only way around this is to hack the source to rename libmctc-lib.a into # something like libmctc_lib.a ... # # Suspect that this part of CASTEP hasn't been tested in some time. # ######################################### #echo "" #echo "4.8 Build the CASTEP Python interface" #echo "=====================================" #make castep_python #pip3 install -e obj/linux_x86_64_gfortran10--mpi/ #make check-python # Production binaries don't need extra debug # information - strip it to make them smaller cd echo "" echo "5. Removing CASTEP debugging symbols" echo "====================================" /bin/file /opt/castep/bin/* | grep "ELF 64-bit" | awk -F: '{print $1}' | while read BIN_FILE do strip -g $BIN_FILE done # Remove all src packages echo "" echo "6. Cleaning up downloaded src tree" echo "==================================" rm -rf /src pip3 cache purge echo "" echo "7. All done" %environment export PYTHONPATH=/opt/castep/lib/python3.12/dist-packages export PATH=/opt/castep/bin:/opt/fftw3/bin:/opt/OpenBLAS/bin:/usr/bin:/usr/local/bin:/usr/sbin:/usr/local/sbin:$PATH export LD_LIBRARY_PATH=/opt/OpenBLAS/lib:/opt/fftw3/lib:/opt/castep/lib:/opt/lapack/lib export CFLAGS="-O3 -march=znver5 -pipe" export CC=gcc-14 export FC=gfortran-14 export OMPI_CC=gcc-14 %runscript ls /opt/castep/bin ---- [[:advanced:software|Back to Software]]