====== CCP-EM Suite ... and related tools ====== > CCPEM - Collaborative Computational Project for Electron cryo-Microscopy CCPEM, Doppio, Relion and //other// tools are often used together. We are providing these tools in a single, self-contained [[advanced:containers|container]] runtime environment for the users of our Comet HPC facility. This wiki page demonstrates how to find and use this software on Comet. * For more details on CCP-EM or Doppio: https://www.ccpem.ac.uk/ Many of the tools included with CCPEM, Doppio and Relion can take advantage of GPU-based acceleration, and you can use the [[started:comet_resources|GPU partitions on Comet]] to run these. Other tools are CPU only and can be run on the standard compute resources. We refer to all of the applications installed alongside CCP-EM as //CCP-EM and tools// for simplicity. For the full list of all individual, installed applications, please consult the table below. ---- ===== Software Packages Included ===== The CCP-EM Suite container includes the following tools and applications. ^ Title ^ Available From ^ Acceleration ^ Link ^ | TEMPy2.0 | 2026.03 | | https://tempy.topf-group.com/ | | RIBFIND2.0 | 2026.03 | | https://ribfind.topf-group.com/download | | EMDA | 2026.03 | | https://github.com/ranganawarshamanage/emda2 | | CheckMySequence | 2026.03 | | https://gitlab.com/gchojnowski/checkmysequence | | findMySequence | 2026.03 | | https://gitlab.com/gchojnowski/findmysequence | | doubleHelix | 2026.03 | | https://gitlab.com/gchojnowski/doublehelix | | Locscale | 2026.03 | | https://cryotud.github.io/locscale/ | | EMDB Validation Analysis | 2026.03 | | https://www.ebi.ac.uk/emdb/va/ | | MetalCoord | 2026.03 | | https://github.com/Lekaveh/MetalCoordAnalysis | | TEMPy-REFF | 2026.03 | GPU | https://www.topf-group.org/tempy-reff | | Topaz | 2026.03 | GPU | https://github.com/3dem/topaz | | CryoDANN | N/A | GPU | https://gitlab.com/ccpem/cryodann | | CryoVAE | N/A | GPU | https://gitlab.com/ccpem/cryovae | | CryoDRGN | 2026.03 | GPU | https://pypi.org/project/cryodrgn/ | | CTFFind 4 | 2026.03 | | https://grigoriefflab.umassmed.edu/ctffind4 | | UCSF MotionCor3 | 2026.03 | GPU | https://emcore.ucsf.edu/ucsf-software | | Resmap | 2026.03 | | https://resmap.sourceforge.net/ | | Relion | 2026.03 | GPU | https://relion.readthedocs.io | | Modeller | 2026.03 | | https://salilab.org/modeller/ | | CCP-EM v1 | 2026.03 | | https://www.ccpem.ac.uk/ | | CCP4 | 2026.03 | | https://www.ccp4.ac.uk/ | | Doppio (aka CCP-EM v2) | 2026.03 | | https://www.ccpem.ac.uk/software/downloads/ | | Python | 2026.03 | | https://www.python.org/ | | gcc, g++, gfortran | 2026.03 | | https://gcc.gnu.org/ | | ARP/wARP | 2026.03 | | https://web.archive.org/web/20231004200010/https://www.embl-hamburg.de/ARP/Manual/UserGuide8.0.html | **Note:** * The //Available From// column indicates the //first// release of the container image that the tool was released in. * //CryoDANN// and //CryoVAE// currently have a version mis-match error in the published conda environments which means that they cannot be installed. ===== Python Modules Included ===== In addition to the standalone applications listed above, the following Python modules are installed - i.e. they are available if you use python3 installed inside the container to run your Python scripts. ^ Module Name ^ Available From ^ Link ^ | yaml | 2026.03 | https://pypi.org/project/PyYAML/ | | typing-extensions | 2026.03 | https://pypi.org/project/typing-extensions/ | | torch | 2026.03 | https://github.com/pytorch/pytorch | | torchvision | 2026.03 | https://github.com/pytorch/vision | | tqdm | 2026.03 | https://github.com/tqdm/tqdm | | mrcfile | 2026.03 | https://pypi.org/project/mrcfile/ | | starfile | 2026.03 | https://pypi.org/project/starfile/ | | loguru | 2026.03 | https://pypi.org/project/loguru/ | | scikit-learn | 2026.03 | https://scikit-learn.org/stable/ | | umap-learn | 2026.03 | https://pypi.org/project/umap-learn/ | | matplotlib | 2026.03 | https://matplotlib.org/ | | morphosamplers | 2026.03 | https://github.com/kevinyamauchi/morphosamplers | | pydantic | 2026.03 | https://docs.pydantic.dev/latest/ | | napari[all] | 2026.03 | https://github.com/napari/napari | | tsnecuda | 2026.03 | https://github.com/CannyLab/tsne-cuda | | PyQt5 | 2026.03 | https://pypi.org/project/PyQt5/ | | typer | 2026.03 | https://typer.tiangolo.com/ | | click | 2026.03 | https://click.palletsprojects.com/en/stable/ | | biopython | 2026.03 | https://biopython.org/ | | fastcluster | 2026.03 | https://pypi.org/project/fastcluster/ | | seaborn | 2026.03 | https://seaborn.pydata.org/ | | dill | 2026.03 | https://pypi.org/project/dill/ | | numpy | 2026.03 | https://numpy.org/ | | scipy | 2026.03 | https://scipy.org/ | | relion-classranker | 2026.03 | https://github.com/3dem/relion-classranker | | relion-blush | 2026.03 | https://github.com/3dem/relion-blush | | DynaMight | 2026.03 | https://github.com/3dem/DynaMight | | topaz | 2026.03 | https://github.com/3dem/topaz | | model-angelo | 2026.03 | https://github.com/3dem/model-angelo | This list only includes modules which have been explicitly installed. The standard //Python// built-ins are still available: ''sqlite'', ''json'', etc. ---- ===== Running CCP-EM Suite on Comet ===== The CCP-EM tools on Comet are provided as an [[advanced:apptainer|Apptainer container environment]] - please read the Apptainer guide first. The CCP-EM container images are stored in the following location: * ''/nobackup/shared/containers/ccpemsuite.2026.03.sif'' - this is the filesystem image with all of the software contained inside You do **not** need to make a copy of this file - it should be left inside the //containers// directory. You will //also// find a matching shell script in the same directory: * ''/nobackup/shared/containers/ccpemsuite.2026.03.sh'' To run any of the CCP-EM tools and applications listed on this page, simply source the shell script: $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh You can then use a helper command ''container.run'' to run commands inside the container, without needing pass huge numbers of parameters to the ''apptainer exec'' command - this automatically adds bind arguments to let you access ''/scratch'' and ''/nobackup'' inside the container, as well as make available Nvidia GPU hardware if you have assigned one via Slurm. For example to run ''relion'' from inside the container image: $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run relion -h You only need to source the shell script once, either at the start of your Comet session, if working interactively, or at the top of your sbatch file if submitting a job. Although we may list a //specific// version of the container and helper files - such as **ccpemsuite.2026.03**, you should check the directory to ensure that there is not a newer version available, just run: ''ls /nobackup/shared/containers'' and it will show any other versions that we have published since. ---- ===== ARP/wARP ===== The use of the web-submission aspects of ARP/wARP is outside the scope of this guide, however the command line tools work as expected. ARP/wARP is installed at ''/opt/arp_warp_8.0'' and all of the tools/commands/scripts in the list below are included: 50_ligands_noH.dat check_refmacversion.sh.bak identify_ligand_multi.sh.bak publ_classic_phases.sh.bak AAbis.XYZ check_sequence.sh identify_partial.sh publ_ligand.sh AAbis_extension.XYZ check_sequence.sh.bak identify_partial.sh.bak publ_ligand.sh.bak ARP_logo_8.0_verysmall216.gif chkdens incl_check_filename.csh publ_loopy.sh ARP_logo_small.gif chkdens_average_deviations incl_check_filename.csh.bak publ_loopy.sh.bak M_ligandbuild chkdens_average_deviations_multi lig_M_align.sh publ_nucleotides.sh NCStry chkseq lig_M_align.sh.bak publ_nucleotides.sh.bak OpenAstexViewer.jar cif2geo ligaline publ_solvent.sh PNSext_ncsrestrainer cif2geo_noH ligand_clusters.txt publ_solvent.sh.bak Restraints.jar cif2pdb ligand_sort read VERSION cleanup_pdb.pl ligandbuild refine_sort all_ligands.pdb crlf likelihood_1.tbl resetprotsize.sh all_ligands_H.pdb density_partial_screen likelihood_2.tbl resetprotsize.sh.bak all_ligands_features.dat density_shape_screen loop rewrite_pdb arp_warp dipcheck loopRamachandran.tab sdf2pdb arpem dipcheck.README loopfit set_1200_2C_my_0.3b_cos_log.llh arpipc_send dipcheck_example.log mapmod set_1200_2N_my_0.3b_cos_log.llh arpnavigator dipcheck_example.pdb mapplus smi2sdf arpnavigator_about.bmp dips_compact.dat mapread snow arpnavigator_help.txt dips_uniform_compact.dat md5only sse2_detection arpnavigator_splash.bmp dnarp mengine syminfo.lib arpwarp_setup_base.bash dnarp_assemble mergefiles symmetry_shift arpwarp_setup_base.csh editrc.awk mir_atom_count tcsh.macosx arpwarp_setup_base.csh.bak editsu.awk mmff94.prm warp_albe.sh arpwarp_setup_base.zsh elal mmxconst.prm warp_albe.sh.bak auto_albe.sh elal.lib mol2pdb warp_build.sh auto_albe.sh.bak energy_all mtzlabels warp_build.sh.bak auto_em.sh extension ncs_arrange warp_fft.sh auto_ligand.sh find_ccp4.sh pca warp_fft.sh.bak auto_ligand.sh.bak find_ccp4.sh.bak pdbrenumber warp_freeset.sh auto_nuce.sh find_fast.sh peaksearch warp_freeset.sh.bak auto_nuce.sh.bak find_fast.sh.bak pept warp_ligand.sh auto_solvent.sh find_output_cluster peptcomp warp_ligand.sh.bak auto_solvent.sh.bak find_output_cluster2 percent_cutter.pl warp_loopy.sh auto_tracing.sh fragdmat ph_error warp_loopy.sh.bak auto_tracing.sh.bak gasteiger ph_rms warp_ncsadjust.sh autopar6.sh gaussm pns_extender.pl warp_ncsadjust.sh.bak autopar6.sh.bak geometriser po4_peaks.sh warp_nuce.sh bfactor geometriser.sh po4_peaks.sh.bak warp_nuce.sh.bak binning geometriser.sh.bak print_warning.sh warp_refmac.sh calc_charges_prot helix_clean print_warning.sh.bak warp_refmac.sh.bak check_java.sh hgen_D publ_albe.sh warp_refmac_restraints.sh check_java.sh.bak hmain publ_albe.sh.bak warp_refmac_restraints.sh.bak check_programs.sh hmain_dens.dat publ_classic_model.sh warp_solvent.sh check_programs.sh.bak hmain_dens_dna.dat publ_classic_model.sh.bak warp_solvent.sh.bak check_prot_remove.sh hmain_table.dat publ_classic_nobuilding.sh warp_tracing.sh check_prot_remove.sh.bak hmain_table_dna.dat publ_classic_nobuilding.sh.bak warp_tracing.sh.bak check_refmacversion.sh identify_ligand_multi.sh publ_classic_phases.sh watercomp Any of the listed commands can be be started using the ''container.run'' helper, for example the ''auto_tracing.sh'' tool: $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run auto_tracing.sh Usage: auto_tracing.sh \ datafile {mtzfile} \ [albe {1 to invoke albe, default is 0 for resol < 2.7A, else 1) }] \ [buildingcycles {number_of_building_cycles (default is 10) }] \ [cgr {number_of_NCS_copies (if seqin is provided, default is 1) }] \ [compareto {PDB_file_for_comparison}] \ [fbest {weighted_amplitude_label}] [phibest {phibest_label}] [fom {fom_label}] \ ... ... $ You do **not** need to initialise ARP/wARP, or ''source'' any shell files - the container is already pre-configured with all of ARP/wARP ready to run. * https://arp-warp.org/ (Note: most links from this page not working as of March 2026) * https://web.archive.org/web/20231004200010/https://www.embl-hamburg.de/ARP/Manual/UserGuide8.0.html ---- ===== CCP-EM v1 ===== The older CCP-EM v1 distribution is installed within the container but the applications/tools within are //not set up to be used without additional work//. Many of the commands in CCP-EM v1 are //also// in Doppio/CCP-EM v2, so would clash if they were added to the default runtime ''$PATH''. You may still access the older CCP-EM v1 tools from the following directory //inside// the container: * ''/opt/ccpem-20221108/bin'' You will need to explicitly run them as follows, for example to run ''cryoEF'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run /opt/ccpem-20221108/bin/cryoEF Most users are __recommended__ to move to Doppio or the other tools included in this container, as development/support for CCP-EM v1 //ceased// in 2022. * https://www.ccpem.ac.uk/software ---- ===== CCP-EM v2 ===== See [[#doppio_aka_ccp-em_v2|Doppio]] ---- ===== CCP4 ===== CCP4 is installed at ''/opt/ccp4-9'' and all of the tools/commands/scripts in the list below are included: ARCIMBOLDO_BORGES dials.split_experiments modelcraft-contents ARCIMBOLDO_LITE dials.split_still_data modelcraft-copies ARCIMBOLDO_SHREDDER dials.spot_counts_per_image molprobity.barbed_wire_analysis JLigand.jar dials.spot_resolution_shells molprobity.cablam JLogGraph.jar dials.ssx_index molprobity.cbetadev LM14.einsle dials.ssx_integrate molprobity.chiral_validation OpenAstexViewer.jar dials.stereographic_projection molprobity.clashscore PdbView dials.stills_process molprobity.clashscore2 TMalign dials.symmetry molprobity.kinemage TMscore dials.two_theta_offset molprobity.molprobity abs dials.two_theta_refine molprobity.mp_validate_bonds acedrg dials.unit_cell_histogram molprobity.omegalyze acorn dials.version molprobity.probe act diffBragg.completeF molprobity.probescore_ligand add_heavy diffBragg.errors molprobity.ramalyze afro diffBragg.estimate_Ncells_Eta molprobity.reduce aimless diffBragg.filt_refls molprobity.rna_validate align_DB diffBragg.geometry_refiner molprobity.rna_validate_bonds almn diffBragg.hopper_process molprobity.rna_validate_puckers alt_sg_list diffBragg.integrate molprobity.rna_validate_suites amore diffBragg.make_input_file molprobity.rotalyze ample diffBragg.pred_offsets molprobity.suitename angles diffBragg.shoebx molprobity.suitename_old anisoanl diffBragg.spectra molprobity.undowser2_validation annlib.python diffBragg.stills_process molprobity.undowser_validation annlib.show_build_path diffBragg.update_stage1_phil molrep annlib.show_dist_paths difres mpi_fxs_launch anode dimple mrbump areaimol distang mrparse arpwarp.source-csh distl.find_active_area mtz2cif arpwarp.source-sh distl.image_viewer mtz2hkl asc2p84 distl.mp_spotfinder_server_read_file mtz2sca astex_viewer distl.signal_strength mtz2various auspex distl.sweep_strength mtzMADmod avs2ps distl.thin_client mtzdiff axissearch dm mtzdump balbes dmmulti mtzfix balls domain2chain mtzinfo baverage doser mtzmnf binsort doubleHelix mtztona4 blend dtrek2mtz mtzutils bltsh dtrek2scala multicomb bltsh24 dui2 na4tomtz bltwish dui2_client nautilus_pipeline bltwish24 dui2_login ncont bones2pdb dui2_multi_user_server ncsmask boost.gcd dui2_old_stable node boost.lcm dui2_server normal3d boost.python dxtbx.any2nexus npm boost.show_build_path dxtbx.depends_on npo boost.show_dist_paths dxtbx.detector_superpose npx boost_adaptbx.divide_by_zero dxtbx.display_parallax_correction oasis boost_adaptbx.double_overflow dxtbx.dlsnxs2cbf omit boost_adaptbx.inexact dxtbx.image2pickle omptbx.python boost_adaptbx.int_overflow dxtbx.image_average omptbx.show_build_path boost_adaptbx.nested_cpp_loops_with_check_signals dxtbx.install_format omptbx.show_dist_paths boost_adaptbx.segmentation_fault dxtbx.overload othercell boost_adaptbx.show_environ_usage dxtbx.plot_detector_models overlapmap boost_adaptbx.show_platform_info dxtbx.print_header p842asc bp3 dxtbx.print_matching_images pairef buccaneer_pipeline dxtbx.python pairef-gui bulking dxtbx.radial_average pandda.analyse cad dxtbx.saturation pandda.export caniso dxtbx.show_build_path pandda.inspect cavenv dxtbx.show_dist_paths pandda.ipython cbf.dump dxtbx.show_mask_info pandda.setup cbflib.python dxtbx.show_matching_formats pandemic.adp cbflib.show_build_path dxtbx.show_registry pcre2-config cbflib.show_dist_paths dyndom pdb2cif cbuccaneer ecalc pdb2ins cbuccaneerNN edstats pdb2s ccontacts edstats.pl pdb2to3 ccp4-python ens.hopper pdb_extract ccp4.setup-csh extends pdb_extract_sf ccp4.setup-csh.bak9350 extract pdb_merge ccp4.setup-sh f2cif pdb_pack ccp4.setup-sh.bak9350 f2mtz pdbcur ccp4.setup-sh.old fable.cout pdbdeposition ccp4_pipeline_simple fable.fem_include_search_paths pdbdiff ccp4cloud-cmd fable.insert_write_at_start_of_each_procedure pdbhtf ccp4cloud-config fable.python pdbset ccp4cloud-local fable.read peakmax ccp4cloud-remote fable.show_build_path percent-rank.pl ccp4i fable.show_calls phaser ccp4i2 fable.show_dist_paths phaser.MRage ccp4i2-update fable.split phaser.MRage.solutions ccp4i_condorsub feckless phaser.abs_convert_hkl_to_mtz ccp4io.python fffear phaser.abs_extract_shelx_refinement_files ccp4io.show_build_path ffjoin phaser.abs_merge_shelx_hkl ccp4io.show_dist_paths fft phaser.abs_reformat_to_sites ccp4ish fhscal phaser.abs_run_phaser ccp4iwish find-blobs phaser.abs_split_dataset ccp4mg findligand phaser.benchmark_basescore_suite ccp4sm findmysequence phaser.benchmark_basescore_test ccp4um findncs phaser.benchmark_best_superposition cctbx.HKLinfo findwaters phaser.benchmark_calculate_mr_statistics cctbx.HKLviewer fragon phaser.benchmark_create_alignment cctbx.brehm_diederichs frame.unpickler phaser.benchmark_create_case cctbx.cod_select_and_pickle freerflag phaser.benchmark_create_model cctbx.data_viewer fsearch phaser.benchmark_create_test cctbx.eltbx.show_fp_fdp gcx phaser.benchmark_error_prediction_suite cctbx.euclidean_model_matching gemmi phaser.benchmark_error_prediction_test cctbx.image_viewer gensym phaser.benchmark_evaluate_suite cctbx.integration_pickle_viewer geomcalc phaser.benchmark_evaluate_test cctbx.loggraph gesamt phaser.benchmark_molecule_replacement cctbx.make_install_HKLviewer_ChimeraX get-residue-stats.pl phaser.benchmark_score_error_estimate cctbx.multiplicity_viewer get_chain phaser.benchmark_score_error_estimate_suite cctbx.patterson_map get_heavy phaser.benchmark_score_suite cctbx.plot_anomalous_scattering get_structure_DB phaser.benchmark_score_test cctbx.precession_view get_trans phaser.benchmark_sculpt_model cctbx.python getax phaser.domain_analysis cctbx.riso giant.cluster_datasets phaser.ensembler cctbx.show_build_path giant.consolidate_cifs phaser.ensembler_new cctbx.show_dist_paths giant.make_restraints phaser.extract_alignments cctbx.show_systematic_absences giant.mass_refine phaser.famos cctbx.small_cell_index giant.merge_conformations phaser.filter_anomalous_outliers cctbx.small_cell_process giant.prepare_datasets phaser.find_alt_orig_sym_mate cctbx.space_group_info giant.quick_refine phaser.find_anomalous_substructure cctbx.structure_factor_timings giant.score_model phaser.gyre_and_gimble cctbx.subgroups giant.split_conformations phaser.nightly_regression_test_report cctbx.suitename giant.swerp phaser.nma_perturb cctbx.suitename_old gltbx.python phaser.phassade cctbx.version gltbx.show_build_path phaser.phil_as_xml cctbx.visualize_r_factors gltbx.show_dist_paths phaser.process_defaults cctbx.wavelength_units gltbx.try_imports phaser.proq2_error_estimation cctbx.web.install gperf phaser.proq2_pdb_to_sculptor_error_file cctbx.wxpython havecs phaser.python cctbx.xfel hbond phaser.rigid_body_divide cctbx.xfel.candidate_cells header2matr phaser.sceds cctbx.xfel.detector_congruence helixang phaser.sculptor cctbx.xfel.detector_residuals hgen phaser.sculptor_new cctbx.xfel.detector_shifts hklplot phaser.show_build_path cctbx.xfel.drift hklview phaser.show_dist_paths cctbx.xfel.ensemble_refinement_pipeline hopper phaser.simple_homology_model cctbx.xfel.experiment_residuals i2run phaser.unit cctbx.xfel.filter_experiments_by_rmsd icoefl phistats cctbx.xfel.frame_extractor icu-config pisa cctbx.xfel.h5_average idiffdisp pixeltool cctbx.xfel.jungfrau_metrics imosflm platonyzer cctbx.xfel.merge import_serial pltdev cctbx.xfel.mpi_integrate iota.double_gauss_visualizer pmf cctbx.xfel.mpi_submit iotbx.blast_pdb pointless cctbx.xfel.plot_reflection_stats iotbx.build_phil_cache polarrfn cctbx.xfel.plot_run_stats_from_experiments iotbx.cbf_read postref cctbx.xfel.plot_run_stats_from_stats_pickle iotbx.cif.as_miller_arrays prasa cctbx.xfel.plot_spotfinder_stats_from_stats_pickle iotbx.cif.as_xray_structure prelyscar_ccp4.pl cctbx.xfel.plot_uc_cloud_from_experiments iotbx.cif.validate prep_bulk cctbx.xfel.powder_from_spots iotbx.cif_as_mtz prime cctbx.xfel.preference iotbx.cif_as_pdb prime.change_basis cctbx.xfel.process iotbx.cns.reflection_reader prime.change_path_indexing_ambiguity cctbx.xfel.recompute_mosaicity iotbx.cns.sdb_reader prime.cluster_isoform cctbx.xfel.show_spot_separation iotbx.cns.transfer_crystal_symmetry prime.compare_solution_pickles cctbx.xfel.small_cell.cake_plot iotbx.crystal_symmetry_from_any prime.comparestats cctbx.xfel.small_cell.cake_plot_prep iotbx.dano_as_amplitudes prime.explore_twin_operators cctbx.xfel.small_cell_process iotbx.distance_least_squares prime.frame_extractor cctbx.xfel.stripe_experiment iotbx.dtrek.to_cns prime.iotacc cctbx.xfel.trumpet_plots iotbx.emma prime.mpi_run cctbx.xfel.ui_server iotbx.export_scalepack_unmerged prime.mpi_scale cctbx.xfel.upload_mtz iotbx.fetch_pdb prime.plot_energy_spectrum cctbx.xfel.weather iotbx.file_reader prime.plotstats cctbx.xfel.xtc_dump iotbx.french_wilson prime.postrefine cctbx.xfel.xtc_process iotbx.get_pubmed_citation prime.print_integration_pickle cctbx_regression.test_nightly iotbx.lattice_symmetry prime.python cecalc iotbx.merging_statistics prime.r_mtz_mtz cextend iotbx.mtz.dump prime.ray_trace cfft iotbx.pdb.add_conformations prime.run chainsaw iotbx.pdb.as_xray_structure prime.show_build_path check_cell_sg iotbx.pdb.box_around_molecule prime.show_dist_paths check_file_DB iotbx.pdb.flip_symmetric_amino_acids prime.solve_indexing_ambiguity check_zero_res.py iotbx.pdb.get_citation prime.view_fineslices checkmysequence iotbx.pdb.insert_scale_records prime.viewstats chef iotbx.pdb.join_fragment_files privateer chltofom iotbx.pdb.link_as_geometry_restraints_edits probe chooch iotbx.pdb.print_sequence probe.probe cif-diff iotbx.pdb.show_connectivity probe.python cif-grep iotbx.pdb.sort_atoms probe.show_build_path cif-merge iotbx.pdb.split_models probe.show_dist_paths cif-validate iotbx.pdb.superpose_atoms_by_name professs cif2mtz iotbx.pdb.superpose_centers_of_mass prosmart cif2pdb iotbx.pdb_as_fasta prosmart_align cif2xml iotbx.pdb_labels_comparison prosmart_restrain ciftab iotbx.pdb_labels_diff pybind11-config cinvfft iotbx.pdb_remediator pyjob clipper_test iotbx.phil pyqt-bundle cloudrun iotbx.poscar_as_xray_structure qdistancefieldgenerator cluster.42 iotbx.python qtpisa cluster.dials.exec iotbx.r_free_flags_accumulation qtrview cluster.individual_frame_intensity iotbx.r_free_flags_as_cns qtwaylandscanner cluster.intensity_statistics iotbx.r_free_flags_completion_simple r500 cluster.unit_cell iotbx.reflection_file_converter r500ccp4 cluster.visualize_orientations iotbx.reflection_file_editor rabdam cma_es.python iotbx.reflection_file_reader rantan cma_es.show_build_path iotbx.shelx.as_cif rapper cma_es.show_dist_paths iotbx.shelx.as_xray_structure rastep cmakereference iotbx.shelx.dump_as_python rebatch cmapcoeff iotbx.show_build_path reduce cmapcut iotbx.show_dist_paths reduce.python cmaplocal iotbx.show_distances reduce.show_build_path cmodeltoseq iotbx.simple_map_coefficients reduce.show_dist_paths cmtzjoin iotbx.split_data_cif refindex cmtzsplit iotbx.symmetry_search refmac5 cnautilus iotbx.unique_with_biomt refmacat cncsfrommodel iotbx.xplor.map.show_summary reforigin combat ipmosflm reindex comit jligand render compar jspisa repc conkit-convert labelit.png revise conkit-msatool libcheck rfcorr conkit-plot libg ribbon conkit-precision libtbx.add_from_future_import_division rings3d conkit-predict libtbx.analyse_code_comments rods conkit-validate libtbx.any2unix rotamer contact libtbx.assert_line_count rotgen convert2mtz libtbx.assert_not_show_diff rotmat coord_format libtbx.assert_stdin rsps coordconv libtbx.assert_stdin_contains_strings rstats coot libtbx.assert_stdin_does_not_contain_strings rstbx.index coot-mini-rsr libtbx.assert_stdin_ends_with rstbx.python cootbx.python libtbx.bash rstbx.show_build_path cootbx.show_build_path libtbx.bibtex rstbx.show_dist_paths cootbx.show_dist_paths libtbx.bootstrap rstbx.simage.create cparrot libtbx.bundle_as_exe rstbx.simage.explore_completeness cpatterson libtbx.bundle_as_selfx rstbx.simage.solver cphasecombine libtbx.check_libcpp rstbx.simage.wx_display cphasematch libtbx.chunk rwcontents cpirate libtbx.clean_clutter samosa.join cprodrg libtbx.clear_paths samosa.scale crank libtbx.configure sapi create_docs libtbx.create_cctbx_bundle_for_installer save_seg_id cross_validate libtbx.create_install_script_bat sc crossec libtbx.create_installer scala crunch2 libtbx.create_mac_app scaleit crys3d.model_viewer libtbx.create_selfx scalepack2mtz crys3d.python libtbx.create_unzipsfx scitbx.apply_tapenade_hints crys3d.qttbx.view_structure libtbx.create_windows_exe scitbx.generate_error_h crys3d.show_build_path libtbx.csh scitbx.histogram crys3d.show_dist_paths libtbx.dtrace scitbx.python csequins libtbx.easy_qsub scitbx.show_build_path csfcalc libtbx.env_run scitbx.show_dist_paths csheetbend libtbx.epydoc_run scitbx.show_exp_times csigmaa libtbx.exec scitbx.show_sizes csloop libtbx.extract_code_from_txt scitbx.unicode_examples csm libtbx.find_clutter search_DB cspad.cbf_metrology libtbx.find_files seqwt cspad.circular_gain_mask libtbx.find_in_repositories serialtbx.python cspad.detector_congruence libtbx.find_old_style_classes serialtbx.show_build_path cspad.detector_congruence2 libtbx.find_pdb_mmcif_problems serialtbx.show_dist_paths cspad.detector_shifts libtbx.find_reserved_names servalcat cspad.detector_statistics libtbx.find_under_build sf_convert cspad.metrology libtbx.find_untested sfall cspad.quadrants libtbx.find_unused_imports sfcheck cspad.quadrants_cbf libtbx.find_unused_imports_crude sftools csymmatch libtbx.gdb shelx2map ctruncate libtbx.git_avoid_merges shelxc cxi.apply_metrology libtbx.help shelxd cxi.brehm_diederichs libtbx.import_all_ext shelxe cxi.calc_gain_ratio libtbx.import_all_python shelxl cxi.calibdir2cbfheader libtbx.install shelxs cxi.cbfheader2slaccalib libtbx.install_conda shelxt cxi.cspad_average libtbx.latex shredcif cxi.cspad_pinwheel libtbx.line_count sigmaa cxi.detector_format_versions libtbx.list_files simbad cxi.diff libtbx.list_modules simbad-contaminant cxi.display_metrology libtbx.list_source_files_sorted_by_size simbad-database cxi.experiment_json_to_cbf_def libtbx.make_dist simbad-full cxi.fee_view libtbx.make_sphinx_html simbad-lattice cxi.find_highres_shots libtbx.nequal simbad-morda cxi.frame_extractor libtbx.parallel_simple simtbx cxi.gain_map libtbx.path_utility simtbx.diffBragg.hopper cxi.gain_map_cbf libtbx.pdflatex simtbx.diffBragg.stage_two cxi.generate_circular_gain_mask libtbx.phil simtbx.model_img cxi.get_next_trial_id libtbx.precommit simtbx.python cxi.hist_finalise libtbx.prime_factors_of simtbx.show_build_path cxi.image2pickle libtbx.printenv simtbx.show_dist_paths cxi.image_average libtbx.py_compile_all slicendice cxi.index libtbx.python smtbx.absolute_structure cxi.list_db_metadata libtbx.raise_exception_for_testing smtbx.anom_refine cxi.lsf libtbx.refresh smtbx.benchmark_refinement cxi.make_dials_mask libtbx.remote_processing smtbx.masks cxi.make_mar_mask libtbx.remove_tree smtbx.python cxi.make_mask libtbx.remove_unused_imports smtbx.refine cxi.merge libtbx.resource_monitor_plot smtbx.show_build_path cxi.monitor_detectors libtbx.run_pickled_function smtbx.show_dist_paths cxi.monitor_trials libtbx.run_tests_parallel solomon cxi.mpi_average libtbx.scons solution_check cxi.mpi_submit libtbx.seconds_since_the_epoch sortmtz cxi.optical2cbfheader libtbx.sge_available_slots sortwater cxi.or_mask libtbx.sge_qstat_counts sphinx.build cxi.parameters libtbx.sh spicker cxi.pbs libtbx.show_all_on_path spicker_omp cxi.pickle2cbf libtbx.show_bin_path spotfinder.python cxi.pixel_histograms libtbx.show_build_options spotfinder.show_build_path cxi.plotcv libtbx.show_build_path spotfinder.show_dist_paths cxi.plotcv_parse libtbx.show_but_no_repr stereo cxi.postrefine libtbx.show_commands stereo3d cxi.print_pickle libtbx.show_date_and_time stgrid cxi.psana libtbx.show_dist_paths stills.merge cxi.pyana libtbx.show_env stnet cxi.radial_average libtbx.show_environ_usage superpose cxi.slaccalib2cbfheader libtbx.show_full_command_path surface cxi.spots libtbx.show_host_and_user symconv cxi.stream_to_pickle libtbx.show_include_paths symfit cxi.trial_stats libtbx.show_lib_path symop2mat cxi.view libtbx.show_module_dependencies tclsh cxi.xes_finalise libtbx.show_number_of_processors tclsh8.4 cxi.xmerge libtbx.show_numeric_version tffc cxi.xtc_process libtbx.show_numpy_version theseus del_heavy libtbx.show_python_dependencies tlsanl density-fitness libtbx.show_python_sys_executable tlsextract detwin libtbx.show_pythonpath tntbx.python dev.cctbx.xfel.detector_residuals libtbx.show_repository_paths tntbx.show_build_path dev.dials.generate_tutorial_text libtbx.show_tests tntbx.show_dist_paths dev.dials.make_sphinx_html libtbx.show_variable_names topdraw dev.dials.napari_rlv libtbx.start_binary_bundle toplist dev.dials.show_extensions libtbx.start_process topp dev.dials.simple_integrate libtbx.tcsh tortoize dev.dxtbx.debug_memory libtbx.unpack_in_sources tracegen dev.dxtbx.read_sequence libtbx.valgrind tracer dev.xia2.check_mosaic libtbx.wait_for_file truncate dev.xia2.create_mask loggraph unique dev.xia2.file_statistics logview vecref dev.xia2.make_sphinx_html lorestr vecsum dev.xia2.show_mask lsqkab vectors dev.xia2.show_mtz_cells makedict viewhkl dials.align_crystal mama2ccp4 volume dials.anvil_correction map2fs watertidy dials.apply_mask mapdiff watncs dials.assign_experiment_identifiers mapdump watpeak dials.augment_spots mapexchange wilson dials.background mapmask wish dials.check_indexing_symmetry mapreplace wish8.4 dials.cluster_unit_cell mapro worms dials.combine_experiments maprot wxtbx.anomalous_differences_histogram dials.compare_orientation_matrices mapsig wxtbx.inspect_r_free_flags dials.complete_full_sphere mapslicer wxtbx.plan_sad_experiment dials.compute_delta_cchalf maptona4 wxtbx.python dials.convert_to_cbf mat2symop wxtbx.show_build_path dials.correlation_matrix matthews_coef wxtbx.show_dist_paths dials.cosym metalCoord wxtbx.show_file_info dials.create_profile_model mkdssp wxtbx.xtriage dials.damage_analysis mlphare xes.histograms dials.data mmCQL xfel.FEE_average_plot dials.detect_blanks mmtbx.altloc_remediate xfel.auto_submit dials.estimate_gain mmtbx.apply_ncs_to_ligand xfel.fee_calibration dials.estimate_resolution mmtbx.arginine_geometry xfel.get_hit_timestamps dials.export mmtbx.bfactor_plot xfel.merge dials.export_best mmtbx.clashscore2 xfel.overlay_spectra dials.export_bitmaps mmtbx.density_modification xfel.plot_isoform_cloud dials.filter_reflections mmtbx.development.aev xfel.plot_run_stats dials.find_bad_pixels mmtbx.development.electrons xfel.plot_uc_cloud dials.find_hot_pixels mmtbx.development.qscore xfel.print_run_stats dials.find_rotation_axis mmtbx.development.validate_ligands xfel.python dials.find_spots mmtbx.development.water_b_factors xfel.show_build_path dials.find_spots_client mmtbx.distance_difference xfel.show_dist_paths dials.find_spots_server mmtbx.emringer_score xfel.smooth_spectrum dials.frame_orientations mmtbx.estimate_bijvoet_ratio xfel.subtract_background dials.generate_distortion_maps mmtbx.fft xia2 dials.generate_mask mmtbx.find_peaks_holes xia2.add_free_set dials.geometry_viewer mmtbx.find_residue_in_pdb xia2.cluster_analysis dials.goniometer_calibration mmtbx.holton_geometry_validation xia2.compare_merging_stats dials.image_viewer mmtbx.hydrogenate xia2.delta_cc_half dials.import mmtbx.massage_data xia2.get_image_number dials.import_xds mmtbx.matthews xia2.html dials.index mmtbx.model_vs_data xia2.index dials.indexed_as_integrated mmtbx.mon_lib_cif_triage xia2.integrate dials.integrate mmtbx.mp_geo xia2.is_doing dials.merge mmtbx.nonbonded_overlaps xia2.ispyb_json dials.merge_cbf mmtbx.pdb_interpretation xia2.ispyb_xml dials.merge_reflection_lists mmtbx.pdb_statistics xia2.merging_statistics dials.missing_reflections mmtbx.plan_sad_experiment xia2.multiplex dials.model_background mmtbx.prepare_pdb_deposition xia2.npp dials.modify_geometry mmtbx.probe2 xia2.overload dials.plot_Fo_vs_Fc mmtbx.process_predicted_model xia2.plot_multiplicity dials.plot_reflections mmtbx.prune_model xia2.print dials.plot_scan_varying_model mmtbx.python xia2.rebatch dials.plugins mmtbx.quantum_interface xia2.report dials.powder_calibrate mmtbx.rama_z xia2.rescale dials.predict mmtbx.rebuild_cablam_cache xia2.setup dials.python mmtbx.rebuild_rotarama_cache xia2.small_molecule dials.rbs mmtbx.reduce xia2.ssx dials.reciprocal_lattice_viewer mmtbx.reduce2 xia2.ssx_reduce dials.reference_profile_viewer mmtbx.refine_anomalous_substructure xia2.strategy dials.refine mmtbx.ribbons xia2.table1 dials.refine_bravais_settings mmtbx.rigid_bond_test xia2.to_shelx dials.refine_error_model mmtbx.ringer xia2.to_shelxcde dials.reflection_viewer mmtbx.search_pdb_symmetry xmlpatterns dials.reindex mmtbx.select_best_starting_model xmlpatternsvalidator dials.report mmtbx.show_build_path xplot84driver dials.rl_png mmtbx.show_dist_paths xpp.beamcenter dials.rlv mmtbx.show_r_factors_by_shell xpp.experiment_manager dials.rs_mapper mmtbx.ss_idealization xpp.isoform dials.scale mmtbx.super xpp.jobs_sentinel dials.search_beam_position mmtbx.superpose xpp.progress_bar dials.sequence_to_stills mmtbx.table_one xpp.progress_detail dials.shadow_plot mmtbx.undowser2_validation xpp.runs_sentinel dials.show mmtbx.validation_summary xpp.simulate dials.show_build_path mmtbx.verify_mon_lib_data zanuda dials.show_dist_paths mmtbx.xtriage zeroed dials.slice_sequence mmtbx.xtriage_gui znd_subs dials.sort_reflections modelcraft Any of the listed commands can be be started using the ''container.run'' helper, for example the ''pisa'' tool: $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run pisa Protein Interactions, Surfaces and Assemblies (PISA) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ v2.2.0 built on 15-03-2019 with SSM v1.4.0, SRS v1.0.0, MMDB v2.0.24 (C) Eugene Krissinel 2007-2019 .============================================================. ! Use of this program is subject to citing the following ! ! publication: ! ! E. Krissinel and K. Henrick (2007). Inference of ! ! macromolecular assemblies from crystalline state. ! ! J. Mol. Biol. 372, 774-797. ! `============================================================' USAGE: ~~~~~~ 1. Perform PISA analysis of a structure given in PDB or mmCIF file 'coorfile': ... ... $ If you run the ''ccp4i'' application in an X terminal on the [[advanced:software:x11|Linux X11 Desktop]] using our [[advanced:interactive|Open OnDemand service]] instead of from a command line, login node or Slurm batch job, you can //also// access the gui interface of CCP4: {{:advanced:software:ood_desktop.png?400|}} {{:advanced:software:ood_desktop_launch_terminal.png?400|}} {{:advanced:software:ood_desktop_terminal_open.png?400|}} Now, in the terminal, run ''ccp4i'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run ccp4i The **CCP4** desktop application window should then appear: {{:advanced:software:ccp4i_gui.png?600|}} Note: the later ''ccp4i2'' GUI tool does __not__ currently work correctly from within the container environment. You do **not** need to initialise CCP4, or ''source'' any shell files to run any CCP4 commands - the container is already pre-configured with all of CCP4 ready to run. * https://www.ccp4.ac.uk * https://www.ccp4.ac.uk/html/ - Documentation on individual CCP4 commands ---- ===== CheckMySequence / DoubleHelix / FindMySequence ===== CheckMySequence, DoubleHelix and FindMySequence are installed within a //conda// environment under ''/opt/package-ccpem2'' //within// the container, but can be started the same as any other command within the container using ''container.run''. The following commands are available: * ''checkmysequence'' * ''doubleHelix'' * ''findmysequence'' For example, to run ''checkmysequence'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run checkmysequence --help # checkMySequence (1.5.3): a tool for sequence assignment validation in EM and MX models of macromolecules findMySequence 1.0.20 doubleHelix 5.6.3 HMMER is available (from CCP4) INFERNAL is available (from CCP4) Usage: checkMySequence [options] Options: --version show program's version number and exit -h, --help show this help message and exit --test a simple test Required parameters (model, sequences and a map): --modelin=FILENAME PDB/mmCIF model --seqin=FILENAME target sequences in FASTA format --mapin=FILENAME map in CCP4/MRC format --mtzin=FILENAME input SFs in MTZ format --labin=FWT,PHWT MTZ file column names Extra options: --fraglen=VALUE starting length of test-fragments [default: 10/MX 20/EM] --max_fraglen=VALUE maximum length of test-fragments [default: 60] --plot generate graphical report --select=STRING model fragment selection string eg. "chain A and resi 10:50" [default: all] --pdbid=PDB_ID Download and check a model directly from PDB (use --keep to store the data in curent directory) --jsonout=FILENAME output filename with the validation results in json format (machine readable) $ * https://gitlab.com/gchojnowski/checkmysequence ---- ===== CryoDRGN ===== CryoDRGN is installed within a //conda// environment under ''/opt/package-ccpem2'' //within// the container, but can be started the same as any other command within the container using ''container.run''. The following commands are available: * ''cryodrgn'' * ''cryodrgn_utils'' For example, to run ''cryodrgn'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run cryodrgn --help usage: cryodrgn [-h] [--version] {abinit_het,abinit_homo,analyze,analyze_landscape,analyze_landscape_full,backproject_voxel,direct_traversal,downsample,eval_images,eval_vol,filter,graph_traversal,parse_ctf_csparc,parse_ctf_star,parse_pose_csparc,parse_pose_star,pc_traversal,train_nn,train_vae,view_config} ... Commands installed with cryoDRGN options: -h, --help show this help message and exit --version show program's version number and exit Choose a command: {abinit_het,abinit_homo,analyze,analyze_landscape,analyze_landscape_full,backproject_voxel,direct_traversal,downsample,eval_images,eval_vol,filter,graph_traversal,parse_ctf_csparc,parse_ctf_star,parse_pose_csparc,parse_pose_star,pc_traversal,train_nn,train_vae,view_config} $ * https://pypi.org/project/cryodrgn/ ---- ===== CTFFind4 ===== CTFFind4 has been compiled and installed to ''/opt/bin'' inside the container. You can run it as follows: $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run ctffind help Usage: ctffind [--old-school-input] [--old-school-input-ctffind4] [--amplitude-spectrum-input] [--filtered-amplitude-spectrum-input] [--fast] [-j ] [controller_address] [controller_port] [job_code] --old-school-input Pretend this is ctffind3 (for compatibility with old scripts and programs) --old-school-input-ctffind4 Accept parameters from stdin, like ctffind3, but with extra lines for ctffind4-specific options (movie processing and phase shift estimation --amplitude-spectrum-input The input image is an amplitude spectrum, not a real-space image --filtered-amplitude-spectrum-input The input image is filtered (background-subtracted) amplitude spectrum --fast Skip computation of fit statistics as well as spectrum contrast enhancement -j: Desired number of threads. Overrides interactive user input. Is overriden by env var OMP_NUM_THREADS $ You can also find ''ctffind_plot_results.sh'' installed: $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run ctffind_plot_results.sh Usage: ctffind_plot_results.sh output_from_ctffind_avrot.txt $ * https://grigoriefflab.umassmed.edu/ctffind4 ---- ===== Doppio (aka CCP-EM v2) ===== Doppio is installed to ''/opt/doppio_1.5.0'' //within// the container. There are two ways in which the Doppio user interface can be launched; **web interface** or **desktop application**. ==== Desktop Application ==== To run the Doppio desktop application you must start a [[advanced:software:x11|Linux X11 Desktop]] session from our [[advanced:interactive|Open OnDemand service]]. Once you have the Linux desktop running, open the terminal application: {{:advanced:software:ood_desktop.png?400|}} {{:advanced:software:ood_desktop_launch_terminal.png?400|}} {{:advanced:software:ood_desktop_terminal_open.png?400|}} Now, in the terminal, run ''doppio-desktop'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run doppio-desktop The desktop application will start within a few seconds: {{:advanced:software:doppio_gui.png?600|}} ==== Web Application ==== To run the Doppio web interface you must start a [[advanced:software:x11|Linux X11 Desktop]] session from our [[advanced:interactive|Open OnDemand service]]. Once you have the Linux desktop running, open the terminal application: {{:advanced:software:ood_desktop.png?400|}} {{:advanced:software:ood_desktop_launch_terminal.png?400|}} {{:advanced:software:ood_desktop_terminal_open.png?400|}} Now, in the terminal, run ''doppio-web'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run doppio-web You should see the following output in the terminal window: Doppio API token: e27d181083116c8c265ca71322449aa054a5205242097d272fa37e5cc32521cb INFO: Started server process [699731] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://127.0.0.1:44185 (Press CTRL+C to quit) INFO: Doppio is now running. Your web browser should open Doppio automatically. If it doesn't, go to this link: file:///mnt/nfs/home/n1234/.local/share/doppio/doppio-699731-open.html Or you can access Doppio directly using these details: Server URL: http://127.0.0.1:44185/ Access token: e27d181083116c8c265ca71322449aa054a5205242097d272fa37e5cc32521cb Press Ctrl+C to quit this Doppio server Whilst still on the Linux desktop, you can then //either// **Control+Left Click** on the //Server URL// address in the terminal, //or// you can **start Firefox from the application menu** and cut-and-paste the address in. You will be asked to enter the //Access token// to verify your access to the running Doppio server. Remember that if you want to use GPU acceleration within Doppio then you should select a GPU partition and allocate a GPU card at the point you start the Open OnDemand session. The Doppio server //only// runs while your Open OnDemand desktop session is running. We __do not__ support running Doppio continuously, nor is there any supported means to expose the Doppio server to the Campus network or the Internet; you must access it while running the desktop session - Comet is __not__ for running network services. * https://www.ccpem.ac.uk/software/downloads/ ---- ===== EMDA ===== EMDA is installed within a //conda// environment under ''/opt/package-ccpem2'' //within// the container, but can be started the same as any other command within the container using ''container.run''. The following commands are included: * ''emda'' * ''emda2'' For example, to run ''emda'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run emda -h usage: emda [command] [arguments] positional arguments: {info,halffsc,fsc,singlemapfsc,ccmask,mapmask,modelmask,lowpass,power,bfac,resol,half2full,map2mtz,map2mtzfull,transform,mtz2map,resample,resamplemap2map,rcc,bfromcc,mmcc,fcc,mapmodelvalidate,mapmodelfsc,overlay,average,diffmap,applymask,scalemap,bestmap,predfsc,refmac,occ,mirror,model2map,composite,magref,com,fetch,axisrefine,pointgroup,symmetrise,rebox,residuecc} optional arguments: -h, --help show this help message and exit --version show program's version number and exit $ * https://github.com/ranganawarshamanage/emda2 ---- ===== EMDB Validation Analysis ===== EMDB VA is installed within a //conda// environment under ''/opt/package-ccpem2'' //within// the container, but can be started the same as any other command within the container using ''container.run''. The following commands are included: * ''va'' To run ''va'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run va -h ChimeraX will be used to produce for the surface view. usage: va [-h] [--version] [-f [F [F ...]]] [-pid [PID]] [-hmeven [HMEVEN]] [-hmodd [HMODD]] [-cl [CL]] [-run [RUN [RUN ...]]] [-met [{tomography,twodcrystal,crystallography,singleparticle,single particle,subtomogramaveraging,subtomogram averaging,helical}]] [-s [S]] [-ms [MS [MS ...]]] [-mscl [MSCL [MSCL ...]]] [-p [P]] [-i MODELMAP] [--strdlib [STRDLIB]] [--threeddir [THREEDDIR]] [-m [M]] [-d [D]] [-emdid [EMDID]] [-j [J]] {strudel} ... $ * https://www.ebi.ac.uk/emdb/va/ ---- ===== Locscale ===== Locscale is installed as a //conda// environment within the container by **package-ccpem2** but can be started like any other command using ''container.run''. The following command are available: * ''locscale'' To run ''locscale'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run locscale usage: locscale [-h] [-em EMMAP_PATH | -hm HALFMAP_PATHS HALFMAP_PATHS] [-filter_input] [-ma MASK] [-v] [--print_report] [--report_filename REPORT_FILENAME] [-op OUTPUT_PROCESSING_FILES] [-fdr FDR_THRESHOLD] [-fdr_w FDR_WINDOW_SIZE] [-avg_filter AVERAGING_FILTER_SIZE] [-fdr_f FDR_FILTER] [-th MASK_THRESHOLD] [-model_path MODEL_PATH] [--use_low_context_model] [-bs BATCH_SIZE] [-gpus GPU_IDS [GPU_IDS ...]] [-cube_size CUBE_SIZE] [-s STRIDE] [-apply_fsc_filter] [-o OUTFILE] [-mm MODEL_MAP | -mc MODEL_COORDINATES] [-mres MODEL_RESOLUTION] [-sym SYMMETRY] [-wn WINDOW_SIZE] [-mpi] [-np NUMBER_PROCESSES] [--measure_bfactors] [-ref_it REFMAC_ITERATIONS] [-res REF_RESOLUTION] [-p APIX] [--add_blur ADD_BLUR] [--refmac5_path REFMAC5_PATH] [--cref_pickle CREF_PICKLE] [-cif_info CIF_INFO] [--complete_model] [-avg_w AVERAGING_WINDOW] [--build_using_pseudomodel] [-pm PSEUDOMODEL_METHOD] [-pm_it TOTAL_ITERATIONS] [-dst DISTANCE] [-mw MOLECULAR_WEIGHT] [--activate_pseudomodel] [-smooth SMOOTH_FACTOR] [--boost_secondary_structure BOOST_SECONDARY_STRUCTURE] [--no_reference] [--set_local_bfactor SET_LOCAL_BFACTOR] [--dev_mode] [--skip_refine] {feature_enhance,version,test} ... $ * https://cryotud.github.io/locscale/ ---- ===== MetalCoord ===== MetalCoord is installed as a //conda// environment within the container by **package-ccpem2** but can be started like any other command using ''container.run''. The following command are available: * ''metalCoord'' To run ''metalCoord'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run metalCoord 2026-03-20 10:04:15,008 - MetalCoord - INFO - Logging started. Logging level: 10 usage: metalCoord [-h] [--version] [--no-progress] {update,stats,coord,pdb} ... MetalCoord: Metal coordination analysis. positional arguments: {update,stats,coord,pdb} update Update a cif file. stats Distance and angle statistics. coord List of coordinations. pdb Get list of PDBs containing the ligand. optional arguments: -h, --help show this help message and exit --version show program's version number and exit --no-progress Do not show progress bar. $ * https://github.com/Lekaveh/MetalCoordAnalysis ---- ===== Model Angelo ===== Model Angelo is installed within the container and can be run using the normal ''container.run'' helper method. The installed commands include: * ''model_angelo build'' * ''model_angelo build_no_seq'' For example: $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run model_angelo build --help usage: model_angelo build [-h] --volume-path VOLUME_PATH --protein-fasta PROTEIN_FASTA [--rna-fasta RNA_FASTA] [--dna-fasta DNA_FASTA] [--output-dir OUTPUT_DIR] [--mask-path MASK_PATH] [--device DEVICE] [--config-path CONFIG_PATH] [--model-bundle-name MODEL_BUNDLE_NAME] [--model-bundle-path MODEL_BUNDLE_PATH] [--keep-intermediate-results] ------------------------------------------------------------------------- Welcome to ModelAngelo! Have a look around... This is the build command, so you need: 1) A cryo-EM map .mrc file, passed to --volume-path/--v/-v 2) A FASTA file with all potential amino acid sequences found in the map, passed to --fasta-path/--f/-f You can also provide a mask for the volume using --mask-path/--m/-m You can also input a custom config file with --config-path/--c/-c options: -h, --help show this help message and exit --rna-fasta RNA_FASTA, --rf RNA_FASTA, -rf RNA_FASTA The path to the RNA sequence file --dna-fasta DNA_FASTA, --df DNA_FASTA, -df DNA_FASTA The path to the DNA sequence file ... $ * Further documentation is available at: https://github.com/3dem/model-angelo ---- ===== Modeller ===== Salilabs' Modeller application is installed from their //Debian package (.deb)// and is available from ''/usr/bin''. You can run Modeller scripts as follows: $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run mod10.8 usage: mod10.8 script [...] $ * https://salilab.org/modeller/ * https://salilab.org/modeller/10.8/manual/ - Online documentation ---- ===== Python ===== This is a //standard// version of Python 3.10 as distributed with Ubuntu 22 LTS. It is used to run many of the packages installed inside the container, but you may also choose to use it to run //your own// scripts, especially if you want to make use of the Python modules which are already installed within the container (torch, model angelo, etc). $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run python3 ---- ===== RIBFIND 2 ===== RIBFIND is installed as a //conda// environment within the container by **package-ccpem2** but can be started like any other command using ''container.run''. The following command are available: * ''ribfind'' To run ''ribfind'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run ribfind usage: ribfind [-h] --model MODEL [--dssp DSSP] [--rnaml RNAML] [--protein-cutoff-dist PROTEIN_CONTACT_DISTANCE] [--protein-helix-ratio PROTEIN_HELIX_RATIO] [--protein-dens-cutoff PROTEIN_DENS_CUTOFF] [--protein-minimum-strand-length PROTEIN_MINIMUM_STRAND_LENGTH] [--rna-contact-dist RNA_CONTACT_DISTANCE] [--rna-minimum-strand-length RNA_MINIMUM_STRAND_LENGTH] [--rna-dens-cutoff RNA_DENS_CUTOFF] [--soft-cutoff] [--output-dir OUTPUT_DIR] [--version] $ * https://ribfind.topf-group.com/download ---- ===== Relion 5 ===== Relion is compiled and installed to ''/opt/bin''. You can find all of the following binaries: relion relion_localsym relion_python_tomo_get_particle_poses relion_tomo_bin_stack relion_align_symmetry relion_localsym_mpi relion_python_tomo_import relion_tomo_catalogue_tomos relion_align_tiltseries relion_manualpick relion_python_tomo_pick relion_tomo_compute_FCC relion_align_tiltseries_mpi relion_mask_create relion_python_tomo_view relion_tomo_convert_projections relion_autopick relion_merge_particles relion_python_topaz relion_tomo_dark_erase relion_autopick_mpi relion_motion_refine relion_qsub.csh relion_tomo_discover_motif relion_backproject_2d relion_motion_refine_mpi relion_reconstruct relion_tomo_find_fiducials relion_batchrun relion_movie_reconstruct relion_reconstruct_mpi relion_tomo_find_lattice relion_batchrun_mpi relion_mrc2vtk relion_refine relion_tomo_fit_bfactors relion_class_ranker relion_particle_reposition relion_refine_mpi relion_tomo_import_coordinates relion_convert_star relion_particle_select relion_run_align_tiltseries relion_tomo_local_particle_refine relion_convert_to_tiff relion_particle_subtract relion_run_align_tiltseries_mpi relion_tomo_make_optimisation_set relion_convert_to_tiff_mpi relion_particle_subtract_mpi relion_run_ctffind relion_tomo_powspec relion_ctf_mask_test relion_particle_symmetry_expand relion_run_ctffind_mpi relion_tomo_predict_tilt_series relion_ctf_refine relion_pipeliner relion_run_motioncorr relion_tomo_reconstruct_particle relion_ctf_refine_mpi relion_plot_delocalisation relion_run_motioncorr_mpi relion_tomo_reconstruct_particle_mpi relion_ctf_toolbox relion_postprocess relion_schemegui.py relion_tomo_reconstruct_tomogram relion_demodulate relion_postprocess_mpi relion_schemer relion_tomo_reconstruct_tomogram_mpi relion_display relion_prepare_subtomo relion_stack_create relion_tomo_refine_ctf relion_estimate_gain relion_preprocess relion_star_datablock_ctfdat relion_tomo_refine_ctf_mpi relion_external_reconstruct relion_preprocess_mpi relion_star_datablock_singlefiles relion_tomo_refine_mag relion_filament_selection relion_project relion_star_datablock_stack relion_tomo_split_optics relion_find_tiltpairs relion_pseudotomo_expansion relion_star_handler relion_tomo_subtomo relion_flex_analyse relion_python_blush relion_star_loopheader relion_tomo_subtomo_mpi relion_flex_analyse_mpi relion_python_classranker relion_star_plottable relion_tomo_taper relion_helix_inimodel2d relion_python_dynamight relion_star_printtable relion_tomo_template_pick relion_helix_toolbox relion_python_fetch_weights relion_suggest_tvalue relion_tomo_test relion_helix_vote_classes relion_python_modelangelo relion_tiltpair_plot relion_tomo_tomo_ctf relion_image_handler relion_python_tomo_align_tilt_series relion_tomo_Wiener_divide relion_import relion_python_tomo_denoise relion_tomo_align relion_it.py relion_python_tomo_exclude_tilt_images relion_tomo_align_mpi To run one of the //command line// tools as an example ''relion_refine'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run relion_refine +++ RELION: command line arguments (with defaults for optional ones between parantheses) +++ ====== General options ===== --i () : Input particles (in a star-file) --tomograms () : Star file with the tomograms, in case subtomos are handled as 2D stacks --trajectories () : Star file with the tomogram motion trajectories --ios () : Input tomo optimisation set file. It is used to set --i, --tomograms, --ref or --solvent_mask if they are not provided. Updated output optimiser set is created. --o () : Output rootname --iter (-1) : Maximum number of iterations to perform --tau2_fudge (-1) : Regularisation parameter (values higher than 1 give more weight to the data) --tau2_fudge_scheme () : Tau2 fudge factor updates scheme. Valid values are plain or -step. Where is the deflate factor during initial stage. --K (1) : Number of references to be refined --particle_diameter (-1) : Diameter of the circular mask that will be applied to the experimental images (in Angstroms) ... $ If you run the main ''relion'' application in an X terminal on the [[advanced:software:x11|Linux X11 Desktop]] using our [[advanced:interactive|Open OnDemand service]] instead of from a command line, login node or Slurm batch job, you can //also// access the gui interface of the tool: {{:advanced:software:ood_desktop.png?400|}} {{:advanced:software:ood_desktop_launch_terminal.png?400|}} {{:advanced:software:ood_desktop_terminal_open.png?400|}} Now, in the terminal, run ''relion'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run relion The **Relion** desktop application should appear: {{:advanced:software:relion_gui.png?500|}} As the ''relion'' application interface launches the built-in processing tools, you should ensure that your Open OnDemand session is configured with the necessary resources (CPU cores and RAM). It can also make use of Nvidia GPU cards //if// launched within a GPU partition //and// a GPU card is assigned at the point of starting your session. For //most// purposes we would still normally recommend that you launch any processing via Slurm sbatch job. * https://relion.readthedocs.io/en/latest ---- ===== Resmap ===== Resmap has been compiled and installed to ''/opt/bin'' inside the container. You can run it as follows: $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run resmap --version 1.1.4 $ If you run ''resmap'' from an X terminal from our [[advanced:software:x11|Open OnDemand Linux X11 Desktop]] instead of from a command line, login node or Slurm batch job, you can //also// access the gui interface of the tool: $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run resmap {{:advanced:software:resmap_gui.png?600|}} In addition, we have also provided the following convenience symlinks, all of which will start ''resmap'' in the same way: * ''/opt/bin/resmap'' * ''/opt/bin/resmap.py'' * ''/opt/bin/ResMap'' * ''/opt/bin/ResMap.py'' * https://resmap.sourceforge.net/ ---- ===== TEMPy 2 ===== TEMPy is installed as a //conda// environment within the container by **package-ccpem2** but can be started like any other command using ''container.run''. The following command are available: * ''TEMPy.smoc'' * ''TEMPy.sccc'' * ''TEMPy.scores'' * ''TEMPy.loqfit'' * ''ribfind'' * ''pdbfixer'' For example, to run ''TEMPy.smoc'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run TEMPy.smoc usage: Calculate SMOC scores [-h] -m MAP -p MODEL [MODEL ...] [--model-colors MODEL_COLORS [MODEL_COLORS ...]] [--model-names MODEL_NAMES [MODEL_NAMES ...]] -r RESOLUTION --output-format OUTPUT_FORMATS [OUTPUT_FORMATS ...] [--output-prefix OUTPUT_PREFIX] [--plot-type PLOT_TYPES [PLOT_TYPES ...]] [--plot-residue-range [PLOT_RESIDUE_RANGE ...]] [--plot-normalize PLOT_NORMALIZE] [--plot-bands [PLOT_BANDS ...]] [--plot-unmodelled PLOT_UNMODELLED] [--smoc-window SMOC_WINDOW] [--smoc-method SMOC_METHOD] $ * https://tempy.topf-group.com/ ---- ===== TEMPy-REFF ===== TEMPy-REFF is installed as a //conda// environment within the container by **package-ccpem2** but can be started like any other command using ''container.run''. The following command are available: * ''tempy-reff'' * ''ribfind'' * ''pdbfixer'' * ''pdb4amber'' For example, to run ''tempy-reff'': $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run tempy-reff usage: tempy-reff [-h] --model MODEL --map MAP [--output-dir OUTPUT_DIR] [--overwrite] [--output-cif] [--platform-cpu] [--platform-cuda] [--platform-cuda-device PLATFORM_CUDA_DEVICE] [--platform-opencl] [--platform-opencl-device PLATFORM_OPENCL_DEVICE] [--platform-precision PLATFORM_PRECISION] [--forcefield-amber] [--forcefield-charmm] [--forcefield-additions FORCEFIELD_ADDITIONS [FORCEFIELD_ADDITIONS ...]] [--forcefield-no-solvent] [--rescale-bfactors] [--keep-hydrogens] [--restrain-ribfind] [--restrain-ribfind-dir RESTRAIN_RIBFIND_DIR] [--restrain-rna-angles [RESTRAIN_RNA_ANGLES]] [--restrain-rna-puckers [RESTRAIN_RNA_PUCKERS]] [--restrain-rna-torsions [RESTRAIN_RNA_TORSIONS]] [--restrain-rna-backbone-termini [RESTRAIN_RNA_BACKBONE_TERMINI]] [--fitting-density] [--fitting-gmm] [--fitting-density-k FITTING_DENSITY_K] [--fitting-density-blur FITTING_DENSITY_BLUR] [--fitting-gmm-k FITTING_GMM_K] [--fitting-gmm-init-steps FITTING_GMM_INIT_STEPS] [--fitting-gmm-bfactor-steps FITTING_GMM_BFACTOR_STEPS] [--fitting-gmm-max-bfactor FITTING_GMM_MAX_BFACTOR] [--simulation-isteps SIMULATION_ISTEPS] [--convergence-type {variance,patience}] [--convergence-timeout CONVERGENCE_TIMEOUT] [--convergence-runs CONVERGENCE_RUNS] [--convergence-threshold CONVERGENCE_THRESHOLD] [--convergence-patience CONVERGENCE_PATIENCE] [--metric {ccc,bfactor-ccc,rmsd}] [--resolution RESOLUTION] $ * https://www.topf-group.org/tempy-reff ---- ===== Topaz ===== **Important!** There are **two** versions of ''topaz'' installed within the container. One version (''/usr/local/bin/topaz'') is a dependency of **Relion**, and the other version (''/opt/package-ccpem2/setup_ext_env/conda_wrappers/topaz_env'') is included by **package-ccpem2**. They have **different** version numbers. Choose the __correct__ one for //your// purposes. Topaz as installed as a dependency for **Relion** can be started as follows: $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run topaz --version TOPAZ 0.2.5a $ Topaz as installed as a //conda// environment by **package-ccpem2** can be started as follows: $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run /opt/package-ccpem2/setup_ext_env/conda_wrappers/topaz_env/topaz --version 0.3.18 $ * https://github.com/3dem/topaz ---- ===== UCSF Motioncor 3 ===== MotionCor3 has been compiled with Nvidia CUDA GPU support and is installed to ''/opt/bin'' inside the container. You can run it as follows: $ source /nobackup/shared/containers/ccpemsuite.2026.03.sh $ container.run MotionCor3 -InMrc -InTiff -InEer -InSuffix -InSkips -OutMrc -FmIntFile -ArcDir -FullSum -Gain -Dark -DefectFile ... ... $ In order to use a GPU, you must __allocate__ one in your Slurm job. See our [[started:resource_overview|Resources Overview]] guide for further information. * https://github.com/czimaginginstitute/MotionCor3 * https://emcore.ucsf.edu/ucsf-software ---- ===== Accessing Data ===== As long as you use the ''container.run'' method to launch the applications, you will automatically be able to read and write to files in your ''$HOME'', ''/scratch'' and ''/nobackup'' directories. If you run any of the applications inside the container manually, without using the ''container.run'' helper you will need to use the ''--bind'' argument to ''apptainer'' to ensure that all relevant directories are exposed within the container. Do remember that the container filesystem itself cannot be changed - so you won't be able to write or update to ''/usr/local'', ''/opt'', ''/etc'' or any other internal folders - keep output directories restricted to the three areas listed above. ---- ===== Building CCP-EM Suite on Comet ===== **Important!** This section is intended for RSE HPC staff, or users who are interested in how the software is configured. If you //only// need to //use// the software, stop reading here. This table lists the //major application dependencies// of CCPEM. Most individual libraries are covered by packages included in Ubuntu (the base OS which this container is built on) - the list below is for software //outside// of the Ubuntu repositories. ^ Package ^ Link ^ Wanted By ^ Type ^ Status ^ Notes ^ | Relion | https://relion.readthedocs.io/en/latest/Installation.html | CCP-EM | Mandatory | Installed | The installation of Relion also includes the download of several moderately large data sets which are included in the container (ClassRanker //model//, Blush //model weights//, ModelAngelo //weights//, nucleotides //data//, nucleotides_no_seq, and a language //model//.)\\ This adds approximately 12-13GB to the size of the container.\\ Relion is also built with the optional AMD FFTW3 libraries. | | CCP4 | https://www.ccp4.ac.uk/download/#os=linux | CCP-EM | Mandatory | Installed | | | ARP WARP | https://www.ccp4.ac.uk/download/#os=linux | //Part of CCP4// | Mandatory | Installed | | | package-ccpem2 | https://gitlab.com/ccpem/package-ccpem2 | CCP-EM | Optional | Installed | Provides pre-configured Conda/Mamba environments of additional key tools; TEMPy2.0, RIBFIND2.0, EMDA, CheckMySequence, findMySequence, doubleHelix, Locscale, EMDB Validation Analysis, MetalCoord, TEMPy-REFF, Topaz, CryoDANN/CryoVAE, CryoDRGN. \\ This adds another 30-35GB to the container. | | Modeller | https://salilab.org/modeller/registration.html | CCP-EM | Optional | Installed | Free to download, but an [[https://salilab.org/modeller/registration.html|academic license key]] is required. | | CTFFind4 | https://grigoriefflab.umassmed.edu | Relion | Mandatory | Installed | | | UCSF MotionCor3 | https://github.com/czimaginginstitute/MotionCor3 | Relion | Optional | Installed | | | Resmap | https://downloads.sourceforge.net/project/resmap | Relion | Optional | Installed | Downloaded as a pre-built binary from the project website. | | Miniforge | https://github.com/conda-forge/miniforge | Relion | Optional | Installed | | | Micromamba | https://mamba.readthedocs.io/en/latest/installation/micromamba-installation.html | package-ccpem2 | Mandatory | Installed | This is only installed to allow package--ccpem2 to install and manage environments; it is not intended for general use by users of the container image. | | PyTorch | https://github.com/pytorch/pytorch | Relion | Mandatory | Installed | A specific version is required (2.0.1) by Relion - this only works with Python <= 3.11. | | Torch Vision | https://github.com/pytorch/vision | Relion | Mandatory | Installed | A specific version is required (0.15.2) by Relion - this only works with Python <= 3.11. | | CUDA | https://developer.nvidia.com/cuda-12-8-0-download-archive | Relion, UCSF MotionCor3, PyTorch | Mandatory | Installed | Added from Nvidia repository, as Ubuntu 22 is limited to CUDA 11. | | CUDNN | https://developer.nvidia.com/cudnn | PyTorch | Mandatory | Installed | Added from Nvidia repository, as Ubuntu 22 is limited to CUDA 11. | | Doppio | https://www.ccpem.ac.uk/software/downloads/ | | Optional | Installed | Doppio is a //web interface// to many of the CCP-EM v2 tools. The only means of normally launching and interacting with this toolset is if launching it within an Open OnDemand desktop session on a compute node. | **Ubuntu Linux versions** CCPEM and many of its dependencies have requirements for //very specific// versions of libraries and other runtime components - not too new versions of Python, quite old versions of Torch, even the Nvidia compilers complain if you use too recent a version of GCC etc. For this reason this container was built on an older Ubuntu 22 LTS image, and __not__ Ubuntu 24 LTS. If you try to build this container with Ubuntu 24 and all of its associated newer libraries many of the components listed above **will not build correctly**. === Prerequisites & Notes === * You must download the file ''ccp4-9.0.014-shelx-arpwarp-linux64.tar.gz'' //before// running the container build. This file can be downloaded from https://www.ccp4.ac.uk/download/#os=linux - but it is subject to a license agreement, therefore we have decided to //not// automatically include it in the container build definition. It should be placed in the same directory as the build script and container definition file. It can be //removed// afterwards. * You must obtain a [[https://www.ccpem.ac.uk/software/licences|Academic license]] to install Doppio. Whilst this does //not// need to be entered during install in the form of a key or code, the use of the software is dependent upon the acceptance of this license. * You must obtain a [[https://salilab.org/modeller/registration.html|Academic license key]] for Salilabs Modeller, and paste the key in to the ''KEY_MODELLER'' variable in the container definition file. * If you are building the container then be aware that ''micromamba'' will attempt to write some environment initialisation code your **your** ''$HOME/.bashrc'' - you can either leave this, or remove it afterwards. This is an artifact of the micromamber installer - the container definition duplicates this by changing the main ''/etc/bash.bashrc'' file within the container itself. **Warning!** At least one of the Conda environments included in **package-ccpem2** does not appear to build correctly. The environment in question is **CryoDANN** and the installation throws this error: ## Creating cryodann_env warning libmamba Cache file "/opt/micromamba/pkgs/cache/88ec62ec.json" was modified by another program warning libmamba Cache file "/opt/micromamba/pkgs/cache/eb045dd1.json" was modified by another program warning libmamba You are using 'pip' as an additional package manager. Be aware that packages installed with 'pip' are managed independently from 'conda-forge' channel. ERROR: Ignored the following versions that require a different python version: 1.21.2 Requires-Python >=3.7,<3.11; 1.21.3 Requires-Python >=3.7,<3.11; 1.21.4 Requires-Python >=3.7,<3.11; 1.21.5 Requires-Python >=3.7,<3.11; 1.21.6 Requires-Python >=3.7,<3.11 ERROR: Could not find a version that satisfies the requirement umap (from cryovae) (from versions: none) ERROR: No matching distribution found for umap critical libmamba pip failed to install packages For this reason, **CryoDANN** is __not__ included in the container image. **Build script:** #!/bin/bash IMAGE_DATE=`date +%Y.%m` echo "Loading modules..." module load apptainer echo "" echo "Building container..." export APPTAINER_TMPDIR=/scratch # Where downloaded package files live that are NOT installed from the internet # during building of the container SW_FOLDER=`pwd` # The following packages should exist: # # ccp4-9.0.014-shelx-arpwarp-linux64.tar.gz - from https://www.ccp4.ac.uk/download/#os=linux # if [ -s "$SW_FOLDER/ccp4-9.0.014-shelx-arpwarp-linux64.tar.gz" ] then echo "Found $SW_FOLDER/ccp4-9.0.014-shelx-arpwarp-linux64.tar.gz" else echo "MISSING: $SW_FOLDER/ccp4-9.0.014-shelx-arpwarp-linux64.tar.gz" fi apptainer build --bind $SW_FOLDER:/mnt ccpemsuite.$IMAGE_DATE.sif ccpemsuite.def 2>&1 | tee ccpemsuite.log **Container Definition:** Bootstrap: docker From: ubuntu:jammy #################################################################### # # This is a container environment for CCP-EM and all of its # supporting software. # # For more details: # https://hpc.researchcomputing.ncl.ac.uk/dokuwiki/doku.php?id=advanced:software:ccpemsuite # # This includes the main packages: # =============================== # - CCP-EM : https://www.ccpem.ac.uk/software/downloads/ # - Relion : https://relion.readthedocs.io/en/latest/Installation.html # - CCP4 : https://www.ccp4.ac.uk/download/#os=linux # - package-ccpem2 : https://gitlab.com/ccpem/package-ccpem2 # - Modeller : https://salilab.org/modeller/registration.html # # Dependencies for each package # # Relion 5 # ======== # - Essential : C++, MPI, cmake, CUDA SDK, CTFFIND-4.1, Ghostscript, FLTK, X11, FFTW, libtiff, libpng, pbzip2, xz, zstd, conda/miniforge # - Optional : UCSF MotionCor2, ResMap # # CCP4 # ==== # Must be downloaded before building container - license must be accepted. # 'install script' ran which sets up license agreement and serveral environment variables. # # - Essential : tcsh # - Optional : No listed dependencies # # Package-CCPEM2 # ============== # # - Essential : micromamba # - Optional : # # Modeller # ======== # # - Essential : # - Optional : # # DOPPIO # ====== # # - Essential : fuse, libfuse2, fusermount # - Optional : %post # Prevent interactive prompts export DEBIAN_FRONTEND=noninteractive export KEY_MODELLER=xxxxxx # Where we build software mkdir -p /src mkdir -p /src/zipped # Where we install software mkdir -p /opt mkdir -p /opt/bin mkdir -p /opt/lib mkdir -p /opt/include mkdir -p /opt/share # If building *on* Comet, set AMD_ARCH=1 AMD_ARCH=1 if [ "$AMD_ARCH" = "1" ] then # Compiling on AMD Epyc - full optimisation (slowest build time = best runtime performance) export CFLAGS="-O3 -march=native -pipe" export CXXFLAGS="-O3 -march=native -pipe" else # Generic target - no optimisation (fastest build time = slowest runtime performance) export CFLAGS="" export CXXFLAGS="" fi # Update & install only necessary packages apt-get update # Base stuff everything will need echo "" echo "INSTALL BASE OS PACKAGES" echo "========================" echo "" apt-get install -y aptitude apt-file wget zip git less vim apt-utils autoconf automake build-essential cmake git less unzip vim wget python3 python3-pip # Dependencies for CCP4 echo "" echo "INSTALL OS PACKAGES (CCP4 DEPENDENCIES)" echo "=======================================" echo "" apt-get install -y \ tcsh \ expect \ curl \ qtbase5-dev \ qtbase5-dev-tools # Dependencies for Relion echo "" echo "INSTALL OS PACKAGES (RELION DEPENDENCIES)" echo "=========================================" echo "" apt-get install -y \ libopenmpi-dev \ cmake \ libfltk1.3-dev \ libfftw3-dev \ libtiff-dev \ libpng-dev \ xz-utils \ pbzip2 \ libzstd-dev \ libxft-dev \ libx11-dev \ libwxgtk3.0-gtk3-dev \ wx-common \ wx3.0-headers \ ghostscript \ libssl-dev \ libcupti-dev \ libopenblas-dev \ libatlas-base-dev \ sphinx \ python3-yaml \ python3-typing-extensions # Dependencies for DOPPIO echo "" echo "INSTALL OS PACKAGES (DOPPIO DEPENDENCIES)" echo "=========================================" echo "" apt-get install -y \ fuse \ libfuse2 \ libfuse-dev # Add on CUDA 12.8 SDK and CUDNN echo "" echo "INSTALL NVIDIA CUDA/CUDNN SDK" echo "=============================" echo "" wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb -O /src/zipped/cuda-keyring_1.1-1_all.deb dpkg -i /src/zipped/cuda-keyring_1.1-1_all.deb apt-get update apt-get -y install cuda-toolkit-12-8 pip install nvidia-pyindex pip install nvidia-cudnn cd /usr/local/cuda/lib64/stubs ln -s libcuda.so libcuda.so.1 # Remove any downloaded package files and any Python cached packages - so they don't remain in the built image apt-get clean # Environment variables export PATH=/opt/bin:$PATH:/opt/miniforge3/bin:/usr/local/cuda/bin export LD_LIBRARY_PATH=/opt/lib:$LD_LIBRARY_PATH export CMAKE_POLICY_VERSION_MINIMUM=3.5 export TORCH_HOME=/usr/local/lib/python3.10/dist-packages/torch export CUDACXX=/usr/local/cuda/bin/nvcc export SHELL=/bin/bash ########################################################### # # !!! Non-OS software package installs below this point !!! # ########################################################### ####################################################### # Install CCP4 - and all related tools ####################################################### echo "" echo "INSTALL CCP4" echo "============" cd /opt tar -zxf /mnt/ccp4-9.0.014-shelx-arpwarp-linux64.tar.gz cd /opt/ccp4-9 # Write 'expect' script to handle the CCP4 interactive license prompt echo "#!/usr/bin/expect" >> /opt/bin/ccp4.exp echo 'spawn ./BINARY.setup' >> /opt/bin/ccp4.exp echo 'expect "Do you accept the terms of the license? y/n"' >> /opt/bin/ccp4.exp echo 'send -- "y\r"' >> /opt/bin/ccp4.exp echo 'expect eof' >> /opt/bin/ccp4.exp chmod +x /opt/bin/ccp4.exp # Run CCP4 installer /opt/bin/ccp4.exp # Cannot use aliases in the file, as we are not (always) using bash cp /opt/ccp4-9/bin/ccp4.setup-sh /opt/ccp4-9/bin/ccp4.setup-sh.old cat /opt/ccp4-9/bin/ccp4.setup-sh.old | grep -v ^alias > /opt/ccp4-9/bin/ccp4.setup-sh # Run ARP WARP installer # Needed to find CCP4 install export CCP4_MASTER="/opt" export CCP4=$CCP4_MASTER/"ccp4-9" cd /opt/arp_warp_8.0 ./install.sh ####################################################### # Install Relion 5 - and all related tools ####################################################### echo "" echo "INSTALL RELION" echo "==============" echo "" echo "" echo "INSTALL RELION (CTFFind4)" echo "=========================" echo "" cd /src wget -q "https://grigoriefflab.umassmed.edu/system/tdf?path=ctffind-4.1.14.tar.gz&file=1" -O /src/zipped/ctffind-4.1.14.tar.gz cd /src tar -zxf zipped/ctffind-4.1.14.tar.gz cd ctffind-4.1.14/ ./configure --prefix=/opt make make install echo "" echo "INSTALL RELION (Motioncor)" echo "==========================" echo "" cd /src git clone https://github.com/czimaginginstitute/MotionCor3.git cd MotionCor3 # Patch odd hardcoded CUDA locations in makefile # Apply our own CFLAGS # Also fix PIE bug (https://github.com/czimaginginstitute/MotionCor3/issues/11) cp makefile11 makefile11.old cat makefile11.old | \ sed 's|NVCC = $(CUDAHOME)/bin/nvcc|NVCC = /usr/local/cuda/bin/nvcc|g' | \ sed 's|CUDAINC = $(CUDAHOME)/include|CUDAINC = /usr/local/cuda/include|g' | \ sed 's|CUDALIB = $(CUDAHOME)/lib64|CUDALIB = /usr/local/cuda/lib|g' | \ sed 's|CFLAG =|CFLAG = ${CFLAGS} |g' | \ sed 's|@$(NVCC) -g -G -m64 $(OBJS)|@$(NVCC) -g -G -m64 -Xcompiler -no-pie $(OBJS)|g' > makefile11 make exe -f makefile11 strip -g MotionCor3 cp -v MotionCor3 /opt/bin echo "" echo "INSTALL RELION (Resmap)" echo "=======================" echo "" cd /src wget -q https://downloads.sourceforge.net/project/resmap/ResMap-1.1.4-linux64 -O /src/zipped/ResMap-1.1.4-linux64 cp -v /src/zipped/ResMap-1.1.4-linux64 /opt/bin/ResMap ln -s /opt/bin/ResMap /opt/bin/ResMap.py ln -s /opt/bin/ResMap /opt/bin/resmap ln -s /opt/bin/ResMap /opt/bin/resmap.py chmod 755 /opt/bin/ResMap echo "" echo "INSTALL RELION (Miniforge)" echo "==========================" echo "" wget -q https://github.com/conda-forge/miniforge/releases/download/26.1.0-0/Miniforge3-26.1.0-0-Linux-x86_64.sh -O /src/zipped/Miniforge.sh chmod +x /src/zipped/Miniforge.sh /src/zipped/Miniforge.sh -b -p /opt/miniforge3 conda update -n base -c conda-forge conda # Install Relion python dependencies # These come from relion/environment.yml and would # normally be installed in a conda environment... but we're # already inside a container, so it doesn't make much sense # to do that here. # # Note the following bug for model-angelo (https://github.com/3dem/relion/issues/1322) echo "" echo "INSTALL RELION (Python modules)" echo "===============================" echo "" echo "#Relion requirements torch==2.0.1 torchvision==0.15.2 tqdm==4.65.0 mrcfile==1.4.3 starfile>=0.5.6 loguru==0.7.0 scikit-learn==1.3.0 umap-learn==0.5.3 matplotlib==3.7.2 morphosamplers==0.0.13 pydantic==1.10.18 napari[all]==0.4.18 tsnecuda==3.0.1 PyQt5==5.15.9 typer==0.9.0 click<8.2.0 biopython==1.81 fastcluster==1.2.6 seaborn==0.12.2 dill==0.3.7 numpy==1.26.1 scipy==1.11.2 git+https://github.com/3dem/relion-classranker git+https://github.com/3dem/relion-blush git+https://github.com/3dem/DynaMight git+https://github.com/3dem/topaz git+https://github.com/3dem/model-angelo@9a44a09cbff805b89220d2fbc89af8123926976c" > /src/relion_requirements.txt pip install -r /src/relion_requirements.txt echo "" echo "INSTALL RELION (compile & build)" echo "================================" echo "" cd /src git clone https://github.com/3dem/relion.git cd relion git checkout ver5.0 mkdir -p build cd build if [ "$AMD_ARCH" = "1" ] then FFTW_LIB="-DAMDFFTW=ON" else FFTW_LIB="" fi CMAKE_C_FLAGS="-std=c99 -fopenmp $CFLAGS" CMAKE_CXX_FLAGS="-fPIC -std=c++17 -fopenmp $CXXFLAGS" cmake .. \ $FFTW_LIB -DFORCE_OWN_FFTW=ON \ -DCUDA_ARCH=89 \ -DCMAKE_INSTALL_PREFIX=/opt \ -DPYTHON_EXE_PATH=/usr/bin/python3 \ -DTORCH_HOME_PATH=/usr/local/lib/python3.10/dist-packages/torch make make install ####################################################### # Install package-ccpem2 ####################################################### echo "" echo "INSTALL PACKAGE-CCPEM2" echo "======================" echo "" # Install micromamba echo "" echo "INSTALL PACKAGE-CCPEM2 (MICROMAMBA)" echo "===================================" echo "" # Addeed cert check bypass since this is hosted on a LetsEncrypt host wget --no-check-certificate -q https://micro.mamba.pm/install.sh -O /src/zipped/micromamba.sh chmod +x /src/zipped/micromamba.sh # Write an 'expect' shell script to handle the # micromamba installer question/answer prompts echo "#!/usr/bin/expect" >> /opt/bin/micromamba.exp echo 'spawn /src/zipped/micromamba.sh' >> /opt/bin/micromamba.exp echo 'expect "Micromamba binary folder?"' >> /opt/bin/micromamba.exp echo 'send -- "/opt/bin\r"' >> /opt/bin/micromamba.exp echo 'expect "Init shell (bash)"' >> /opt/bin/micromamba.exp echo 'send -- "y\r"' >> /opt/bin/micromamba.exp echo 'expect "Configure conda-forge?"' >> /opt/bin/micromamba.exp echo 'send -- "y\r"' >> /opt/bin/micromamba.exp echo 'expect "Prefix location?"' >> /opt/bin/micromamba.exp echo 'send -- "/opt/micromamba\r"' >> /opt/bin/micromamba.exp echo 'expect eof' >> /opt/bin/micromamba.exp chmod +x /opt/bin/micromamba.exp /opt/bin/micromamba.exp # Install the ccpem2 packages echo "" echo "INSTALL PACKAGE-CCPEM2" echo "======================" echo "" # This is a huge amount of extra software # installed as conda/mamba environments (and hence loads of duplication...) cd /opt git clone https://gitlab.com/ccpem/package-ccpem2.git cd package-ccpem2/setup_ext_env ./install_doppioext.sh checkmysequence emda tempy tempyreff locscale emdbva metalcoord cryodrgn topaz ######################################################## # Install Modeller ######################################################## echo "" echo "INSTALL MODELLER" echo "================" echo "" # KEY_MODELLER must be set! wget -q https://salilab.org/modeller/10.8/modeller_10.8-1_amd64.deb -O /src/zipped/modeller_10.8-1_amd64.deb dpkg -i /src/zipped/modeller_10.8-1_amd64.deb ######################################################## # Install Doppio ######################################################## echo "" echo "INSTALL DOPPIO/CCP-EM v2" echo "========================" echo "" wget -q https://www.ccpem.ac.uk/downloads/doppio/doppio-linux-1.5.0.sh -O /src/zipped/doppio-linux-1.5.0.sh chmod +x /src/zipped/doppio-linux-1.5.0.sh /src/zipped/doppio-linux-1.5.0.sh -b -m -p /opt/doppio_1.5.0 # Cannot run an .AppImage in a container, as it is not # allowed to fusermount due to security restrictions. # Need to extract the appimage and run from the extracted folder cd /opt/doppio_1.5.0/app ./doppio-desktop.AppImage --appimage-extract mv -v squashfs-root /opt/doppio_1.5.0/app/doppio-desktop-app cd /opt/doppio_1.5.0/app/doppio-desktop-app ln -s AppRun ./doppio-desktop # Now we can just run 'doppio-desktop' instead ######################################################## # Install CCP-EM v1 ######################################################## echo "" echo "INSTALL CCP-EM v1" echo "=================" wget -q https://www.ccpem.ac.uk/downloads/ccpem_v1/ccpem-20221108-linux-x86_64.tar.gz -O /src/zipped/ccpem-20221108-linux-x86_64.tar.gz cd /opt tar -zxf /src/zipped/ccpem-20221108-linux-x86_64.tar.gz cd /opt/ccpem-20221108 # Do not do anything else with CCP-EM v1, as it has commands/binaries inside it which are *also* # included in CCP4 and similar, we do not want to use older binaries from v1! ######################################################## # Create Apptainer runtime environment script ######################################################## echo "" echo "INJECT APPTAINER INIT FILE" echo "==========================" echo "" # NOTE: Creating /.singularity.d/env entries is *bad* # ... but... this is the only way to have shell initialisation # 'stuff' that is in effect both for an 'apptainer shell', as # well as anything which is ran via 'apptainer exec', as # /etc/bash.bashrc is *not* executed in 'apptainer exec' mode. touch /.singularity.d/env/99-ccpem.sh echo '#!/bin/sh' > /.singularity.d/env/99-ccpem.sh # Add micromamba to paths echo 'export MAMBA_EXE="/opt/bin/micromamba"' >> /.singularity.d/env/99-ccpem.sh echo 'export MAMBA_ROOT_PREFIX="/opt/micromamba"' >> /.singularity.d/env/99-ccpem.sh # Add CCP4 to paths echo '. /opt/ccp4-9/bin/ccp4.setup-sh' >> /.singularity.d/env/99-ccpem.sh # Add ARP/WARP to paths echo '. /opt/arp_warp_8.0/arpwarp_setup.bash' >> /.singularity.d/env/99-ccpem.sh # Add micromamba initialisation, but filter out zsh syntax which makes /bin/sh complain /opt/bin/micromamba shell hook --shell bash --root-prefix "$MAMBA_ROOT_PREFIX" | grep -v COMP_LINE >> /.singularity.d/env/99-ccpem.sh # This next line doesn't work as the script assumes bash... but Apptainer doesn't use bash to init the container # See the Package-ccpem2 PATH variable below, as an alternative #echo '. /opt/package-ccpem2/setup_ext_env/setup_doppioext.sh' >> /.singularity.d/env/99-ccpem.sh chmod +x /.singularity.d/env/99-ccpem.sh ######################################################## # Final tidyup ######################################################## echo "" echo "CLEANING UP" echo "===========" echo "" # Remove src tree rm -rf /src # Remove pip caches apt-get clean # Remove any expect scripts used during install rm -f /opt/bin/*.exp # Remove micromamba caches rm -rf /opt/micromamba/pkgs/cache/* %environment # For anything we build after the image is created using the installed compilers # AMD/Comet export CFLAGS="-O3 -march=native -pipe" export CXXFLAGS="-O3 -march=native -pipe" # Development/others export CFLAGS="" export CXXFLAGS="" export PATH=/opt/bin:$PATH:/opt/miniforge3/bin:/usr/local/cuda/bin export LD_LIBRARY_PATH=/opt/lib:/usr/local/lib:$LD_LIBRARY_PATH export TORCH_HOME=/usr/local/lib/python3.10/dist-packages/torch export CUDACXX=/usr/local/cuda/bin/nvcc # CCP4 export CCP4_MASTER="/opt" export CCP4=$CCP4_MASTER/"ccp4-9" export CCP4I_TCLTK=$CCP4/bin export BALBES_ROOT=$CCP4/BALBES export GFORTRAN_UNBUFFERED_PRECONNECTED=Y export CBIN=$CCP4/bin export CLIB=$CCP4/lib export CLIBD=$CCP4/lib/data export CETC=$CCP4/etc export CINCL=$CCP4/include export CHTML=$CCP4/html export CEXAM=$CCP4/examples export CCP4I_TOP=$CCP4/share/ccp4i export MMCIFDIC=$CLIB/ccp4/cif_mmdic.lib export CRANK=$CCP4I_TOP/crank export CLIBD_MON=$CCP4/lib/data/monomers/ export CCP4_HELPDIR=$CCP4/help/ export CCP4_SCR="/scratch/ccp4_$USER" export PDB_EXTRACT=${CCP4}/share export CCP4_OPEN=UNKNOWN export PATH=$CCP4/etc:$CCP4/bin:$PATH export MANPATH=$CCP4/share/man:$MANPATH # Package-ccpem2 export PATH=$PATH:/opt/package-ccpem2/setup_ext_env/conda_wrappers/cryodrgn_env export PATH=$PATH:/opt/package-ccpem2/setup_ext_env/conda_wrappers/topaz_env export PATH=$PATH:/opt/package-ccpem2/setup_ext_env/conda_wrappers/metalcoord_env export PATH=$PATH:/opt/package-ccpem2/setup_ext_env/conda_wrappers/emdbva_env export PATH=$PATH:/opt/package-ccpem2/setup_ext_env/conda_wrappers/locscale_env export PATH=$PATH:/opt/package-ccpem2/setup_ext_env/conda_wrappers/tempyreff_env export PATH=$PATH:/opt/package-ccpem2/setup_ext_env/conda_wrappers/tempy_env export PATH=$PATH:/opt/package-ccpem2/setup_ext_env/conda_wrappers/emda_env export PATH=$PATH:/opt/package-ccpem2/setup_ext_env/conda_wrappers/checkmysequence_env # ARP WARP export warpbin=/opt/arp_warp_8.0/bin/bin-x86_64-Linux export warpdoc=/opt/arp_warp_8.0/manual export PATH=${warpbin}:$PATH # Modeller export KEY_MODELLER=xxxxxx # Micromamba export MAMBA_EXE="/opt/bin/micromamba" export MAMBA_ROOT_PREFIX="/opt/micromamba" # Doppio export PATH=/opt/doppio_1.5.0/app:/opt/doppio_1.5.0/app/doppio-desktop-app:$PATH %runscript **Run Script:** #!/bin/bash module load apptainer IMAGE_NAME=/nobackup/shared/containers/ccpemsuite.2026.03.sif container.run() { # Run a command inside the container... # automatically bind the /scratch and /nobackup dirs # pass through any additional parameters given on the command line apptainer exec --nv --bind /scratch:/scratch --bind /nobackup:/nobackup ${IMAGE_NAME} $@ } ---- [[:advanced:software|Back to Software]]