===== sacct for resource usage ======
see also FAQ [[faq:004]] [[faq:019]] [[faq:027]]


sacct is helpful for seeing resource usage by job e.g.

<code>
$ sacct --user ncb176 --format=JobID,Jobname%50,partition,elapsed,nnodes,ncpus,nodelist  --units=g
</code>

gives output like
<code>
JobID                                                   JobName  Partition    Elapsed   NNodes      NCPUS        NodeList
------------ -------------------------------------------------- ---------- ---------- -------- ---------- ---------------
1144351                                      interactive-Matlab interacti+   00:08:24        1          1      compute001
1144351.bat+                                              batch              00:08:24        1          1      compute001
1144351.ext+                                             extern              00:08:24        1          1      compute001
1144353                                      interactive-Matlab interacti+   00:02:10        1          1      compute001
1144353.bat+                                              batch              00:02:11        1          1      compute001
1144353.ext+                                             extern              00:02:12        1          1      compute001
</code>


=== filter for start and end time range ===
add `-S` and `-E` for start and end dates in format YYYY-MM-DD

<code>
$ sacct --user ncb176 -S 2025-12-01 -E 2025-12-12 --format=JobID,Jobname%50,partition,elapsed,nnodes,ncpus,nodelist  --units=g
</code>
<code>
JobID                                                   JobName  Partition    Elapsed   NNodes      NCPUS        NodeList
------------ -------------------------------------------------- ---------- ---------- -------- ---------- ---------------
1143402                                     interactive-desktop interacti+   00:07:34        1          2      compute030
1143402.bat+                                              batch              00:07:34        1          2      compute030
1143402.ext+                                             extern              00:07:34        1          2      compute030
1143403                                     interactive-desktop interacti+   00:05:04        1          2      compute030
1143403.bat+                                              batch              00:05:05        1          2      compute030
1143403.ext+                                             extern              00:05:06        1          2      compute030
1143660                                     interactive-desktop interacti+   01:00:28        1          2      compute032
1143660.bat+                                              batch              01:00:29        1          2      compute032
1143660.ext+                                             extern              01:00:29        1          2      compute032
1144351                                      interactive-Matlab interacti+   00:08:24        1          1      compute001
1144351.bat+                                              batch              00:08:24        1          1      compute001
1144351.ext+                                             extern              00:08:24        1          1      compute001
1144353                                      interactive-Matlab interacti+   00:02:10        1          1      compute001
1144353.bat+                                              batch              00:02:11        1          1      compute001
1144353.ext+                                             extern              00:02:12        1          1      compute001
</code>

=== checking for pre-empted jobs ===
<code>
sacct -S now-30days -E now -X -p --duplicate -a --state=PREEMPTED
</code>

gave output like
<code>
1147921_13|anal.sh|long_free|comet_kogwas|1|PREEMPTED|0:0|
1149928_2|gpeA|long_free|rockhpc_twnss|10|PREEMPTED|0:0|
1149936_4|gpeA|long_free|rockhpc_twnss|10|PREEMPTED|0:0|
1149936_5|gpeA|long_free|rockhpc_twnss|10|PREEMPTED|0:0|
</code>
while there was an issue with job pre-emption happening on the long_free queue, probably due to mis-configuration.

=== options for formatting output ===
  * ''-p'' = delimited
  * ''sacct -e'' = list all options for ''--format'' (NB this only shows the fields named in the ''--format'' block but can be combined with ''-p'')

<code>
sacct --format="JobID,User,Partition" -S now-30days -E now -X -p --duplicate -a --state=PREEMPTED
</code>
gave output like
<code>
1147921_2|nrajh|long_free|
1147921_13|nrajh|long_free|
1149928_2|nikl|long_free|
1149936_4|nikl|long_free|
1149936_5|nikl|long_free|
</code>


=== options used ===

''%%-D, --duplicates%%''

If Slurm job ids are reset, some job numbers will probably appear more than once in the accounting log file but refer to different jobs. Such jobs can be distinguished by the "submit" time stamp in the data records.
When data for specific jobs are requested with the --jobs option, sacct returns the most recent job with that number. This behavior can be overridden by specifying --duplicates, in which case all records that match the selection criteria will be returned

-----------------
==== a few more Carol has found interesting ====
=== how much is going on on this node? ===
<code>
$ sacct -X --node=compute031 --format="JobID,Partition,Start%20,Timelimit%15,ncpus,ReqMem"  --state=RUNNING
JobID         Partition                Start       Timelimit      NCPUS     ReqMem 
------------ ---------- -------------------- --------------- ---------- ---------- 
1147921_5     long_free  2026-01-05T17:27:13     14-00:00:00          1       100G 
1147921_9     long_free  2026-01-05T17:27:13     14-00:00:00          1       100G 
</code>

=== what's that job using all those cores? ===
<code>
$ sacct -X --job=1155749 --format="JobID,JobName,User,Partition,Start%20,Timelimit%15,ncpus,ReqMem" -p --state=RUNNING
JobID|JobName|User|Partition|Start|Timelimit|NCPUS|ReqMem|
1155749|interactive-RStudio|c2040047|interactive-std_free|2026-01-13T12:05:08|08:00:00|200|60G|
</code>



--------------------
[[faq:index| Back to faq]]