This is an old revision of the document!
The HPC service is shared between many users and it is easy for one person to cause problems for others. You are expected to behave in a considerate way when you use the service.
The HPC facilities are provided to allow Newcastle University staff and students to undertake computational work as part of their teaching or coursework related to academic studies at the University, or as part of research being conducted within Newcastle University. Comet was funded from the UKRI Research Capital Investment Fund https://www.ukri.org/what-we-do/browse-our-areas-of-investment-and-support/research-capital-investment-fund/
It is not permitted to use the facilities for personal gain or for commercial activities unrelated to those which are sanctioned by the University as part of teaching or research.
You may have heard the phrase Trusted Research Environment, here are some descriptions that attempt to explain it:
Trusted research environments (TREs) enable researchers to gain access to data in a safe way.
Trusted Research Environments (TREs) are highly secure and controlled computing environments that allow approved researchers from authorised organisations a safe way to access, store, and analyse sensitive data remotely.
Trusted Research Environments (TRE) are platforms for analysing data which protects privacy.
If any of these descriptions fit the work you intend to carry out, please contact the Research Software Engineering team before you begin, as this facility may not be an appropriate environment for you.
Use of AI Agents on the login nodes is not allowed. This includes using agents from an IDE on your local (e.g. Visual Studio Code) via ssh connection to Comet. Installation of software used to access AI Agents (e.g. Codex CLI, Claude Code) is also not permitted on any Comet filesystem.
Impact on the system:AI Agent use on HPC is a concern to the HPC community at large. Agents operate using the same credentials as human users but with very different behaviour patterns, leading to concerns around security, reproducibility and provenance of code, overloading of filesystems and the scheduler.
Licensing: The University does not hold licenses for any AI Agents other than Microsoft Copilot and personal / Pro licenses do not cover use in the University environment, or provide the protections that a large organisation needs.
Your privacy and sensitive data: AI Agents are hosted by the operating company and when you allow access to your files, you allow that company access to all the files you can read, including hidden and private information; as an individual you do not have the authority to make data agreements with an 3rd party AI company on behalf of the University. You have no control over where this data is stored and usually limited control over how it is used. If the company security is breached then your data can easily be leaked to to criminals.
As a small team supporting HPC at Newcastle University, we don't have the resource to properly regulate coding agent use on HPC or lead the field in a rapidly evolving area. Because widespread use of AI Agents would put at risk fair and secure access to the service for all users, our only option at present is to deny all use.
Users can still achieve benefits by downloading outputs to their local machine and use a coding agent to analyse them there.
Be considerate when submitting jobs via Slurm. If you submit many hundreds of jobs within a short space of time this may prevent other users of the facility from having a fair share of the available resources.
If any user is found abusing the job submission system and preventing others from submitting or running jobs they will be requested to cease submitting new jobs. Their existing backlog of pending jobs may also be purged in order to allow others to continue using the system.
Do not use automation tools to run commands like squeue or sacct or similar in tight loops which can put undue load on the entire scheduler system. Processes found to be abusing calls to the scheduler are subject to be killed automatically to prevent disruption of the entire scheduling system for all other users.
squeue
sacct
The login nodes are for low-intensity, interactive work such as managing your files, compiling code, running short test cases and submitting jobs.
Do not run intensive or long running jobs or attempt to set up port forwards on the login nodes. The login nodes are shared between all users and can easily be disrupted by intensive jobs. Use Slurm to submit jobs to the compute nodes instead or book out an interactive desktop or application via Open OnDemand instead if the login node is not appropriate.
Jobs that cause problems on the login nodes, or otherwise fit in to the above categories of use will be killed.
Processes which are found to be excessively long running for many days, using substantial amounts of memory or causing high CPU load over a sustained period of time will be killed automatically.
No areas on the HPC facilities are intended for long-term data storage and they should not be used for those purposes. Backup and archival of data is outside the scope of any existing services offered by HPC.
If the /nobackup filesystem becomes very full, we reserve the right to remove files belonging to people whose use is excessive or that are old and not in use, in order to allow the system to continue functioning.
/nobackup
Use the local storage on compute nodes ($TMPDIR or /scratch) for large temporary files and intensive I/O during your jobs - writing back to /nobackup often during jobs is both (a) wasteful and (b) disruptive to other jobs.
$TMPDIR
/scratch
Remember that the security of your data is your responsibility. If your work involves data that can identify individuals or that are sensitive for commercial or other reasons, be aware of all relevant legal and commercial requirements and take responsibility for compliance (See TRE, above). Genetic and biometric data are considered sensitive under the EU General Data Protection Regulation (GDPR). See Research Data Management.
Back to Policies & Procedures
Table of Contents
HPC Service
Main Content Sections
Documentation Tools