faq:043 [HPC Support]

sacct for resource usage

see also FAQ 004 019 027

sacct is helpful for seeing resource usage by job e.g.

$ sacct --user ncb176 --format=JobID,Jobname%50,partition,elapsed,nnodes,ncpus,nodelist  --units=g

gives output like

JobID                                                   JobName  Partition    Elapsed   NNodes      NCPUS        NodeList
------------ -------------------------------------------------- ---------- ---------- -------- ---------- ---------------
1144351                                      interactive-Matlab interacti+   00:08:24        1          1      compute001
1144351.bat+                                              batch              00:08:24        1          1      compute001
1144351.ext+                                             extern              00:08:24        1          1      compute001
1144353                                      interactive-Matlab interacti+   00:02:10        1          1      compute001
1144353.bat+                                              batch              00:02:11        1          1      compute001
1144353.ext+                                             extern              00:02:12        1          1      compute001

filter for start and end time range

add `-S` and `-E` for start and end dates in format YYYY-MM-DD

$ sacct --user ncb176 -S 2025-12-01 -E 2025-12-12 --format=JobID,Jobname%50,partition,elapsed,nnodes,ncpus,nodelist  --units=g

JobID                                                   JobName  Partition    Elapsed   NNodes      NCPUS        NodeList
------------ -------------------------------------------------- ---------- ---------- -------- ---------- ---------------
1143402                                     interactive-desktop interacti+   00:07:34        1          2      compute030
1143402.bat+                                              batch              00:07:34        1          2      compute030
1143402.ext+                                             extern              00:07:34        1          2      compute030
1143403                                     interactive-desktop interacti+   00:05:04        1          2      compute030
1143403.bat+                                              batch              00:05:05        1          2      compute030
1143403.ext+                                             extern              00:05:06        1          2      compute030
1143660                                     interactive-desktop interacti+   01:00:28        1          2      compute032
1143660.bat+                                              batch              01:00:29        1          2      compute032
1143660.ext+                                             extern              01:00:29        1          2      compute032
1144351                                      interactive-Matlab interacti+   00:08:24        1          1      compute001
1144351.bat+                                              batch              00:08:24        1          1      compute001
1144351.ext+                                             extern              00:08:24        1          1      compute001
1144353                                      interactive-Matlab interacti+   00:02:10        1          1      compute001
1144353.bat+                                              batch              00:02:11        1          1      compute001
1144353.ext+                                             extern              00:02:12        1          1      compute001

checking for pre-empted jobs

sacct -S now-30days -E now -X -p --duplicate -a --state=PREEMPTED

gave output like

1147921_13|anal.sh|long_free|comet_kogwas|1|PREEMPTED|0:0|
1149928_2|gpeA|long_free|rockhpc_twnss|10|PREEMPTED|0:0|
1149936_4|gpeA|long_free|rockhpc_twnss|10|PREEMPTED|0:0|
1149936_5|gpeA|long_free|rockhpc_twnss|10|PREEMPTED|0:0|

while there was an issue with job pre-emption happening on the long_free queue, probably due to mis-configuration.

options for formatting output

-p = delimited
sacct -e = list all options for –format (NB this only shows the fields named in the –format block but can be combined with -p)

sacct --format="JobID,User,Partition" -S now-30days -E now -X -p --duplicate -a --state=PREEMPTED

gave output like

1147921_2|nrajh|long_free|
1147921_13|nrajh|long_free|
1149928_2|nikl|long_free|
1149936_4|nikl|long_free|
1149936_5|nikl|long_free|

options used

-D, --duplicates

If Slurm job ids are reset, some job numbers will probably appear more than once in the accounting log file but refer to different jobs. Such jobs can be distinguished by the “submit” time stamp in the data records. When data for specific jobs are requested with the –jobs option, sacct returns the most recent job with that number. This behavior can be overridden by specifying –duplicates, in which case all records that match the selection criteria will be returned

a few more Carol has found interesting

how much is going on on this node?

$ sacct -X --node=compute031 --format="JobID,Partition,Start%20,Timelimit%15,ncpus,ReqMem"  --state=RUNNING
JobID         Partition                Start       Timelimit      NCPUS     ReqMem 
------------ ---------- -------------------- --------------- ---------- ---------- 
1147921_5     long_free  2026-01-05T17:27:13     14-00:00:00          1       100G 
1147921_9     long_free  2026-01-05T17:27:13     14-00:00:00          1       100G

what's that job using all those cores?

$ sacct -X --job=1155749 --format="JobID,JobName,User,Partition,Start%20,Timelimit%15,ncpus,ReqMem" -p --state=RUNNING
JobID|JobName|User|Partition|Start|Timelimit|NCPUS|ReqMem|
1155749|interactive-RStudio|c2040047|interactive-std_free|2026-01-13T12:05:08|08:00:00|200|60G|

Back to faq