====== Rclone ======

> Rclone is a command-line program to manage files on cloud storage. It is a feature-rich alternative to cloud vendors' web storage interfaces. Over [[https://rclone.org/#providers|70 cloud storage products]] support rclone including S3  object stores, business & consumer file storage services, as well as standard transfer protocols.
>
> Rclone has powerful cloud equivalents to the unix commands rsync, cp, mv, mount, ls, ncdu, tree, rm, and cat. Rclone's familiar syntax includes shell pipeline support, and --dry-run protection. It is used at the command line, in scripts or via its API.

If you need to interact with Amazon or Google storage from **Rocket** or **Comet**, as part of your data workflow on the HPC facilities, Rclone will let you do that.

   * https://rclone.org/

----

===== Installation =====

Rclone is not installed on our HPC facilities (neither **Rocket** nor **Comet**), but the developer fortunately makes a pre-built, read-to-run version available to download for most compatible Linux systems.

   * https://rclone.org/downloads/

The Rclone download page shows a table of available versions:

{{:advanced:rclone_download_table.png?600|}}

The version we need, to run from within our own personal area(s) on the HPC systems is the **Intel/AMD - 64 Bit** row, and the **Linux** column; __not__ the **.deb** or **.rpm** version - you //cannot// install those versions yourself.

Right click on the download icon of the correct version, and copy the link. In the above case, as of May 2025, it is ''https://downloads.rclone.org/v1.69.1/rclone-v1.69.1-linux-amd64.zip'' - __but this will differ__, and you should always visit the URL above to get the most current link. 

Now open up a SSH connection to the HPC and run the following:

<code lang=shell-session>
$ wget https://downloads.rclone.org/v1.69.1/rclone-v1.69.1-linux-amd64.zip
Saving to: ‘rclone-v1.69.1-linux-amd64.zip’
100%[==========================================================>] 23,314,273  23.9MB/s   in 0.9s
$
</code>

Once the ''.zip'' file has downloaded we should extract it:

<code lang=shell-session>
$ unzip rclone-v1.69.1-linux-amd64.zip
Archive:  rclone-v1.69.1-linux-amd64.zip
   creating: rclone-v1.69.1-linux-amd64/
  inflating: rclone-v1.69.1-linux-amd64/README.html  
  inflating: rclone-v1.69.1-linux-amd64/git-log.txt  
  inflating: rclone-v1.69.1-linux-amd64/rclone  
  inflating: rclone-v1.69.1-linux-amd64/README.txt  
  inflating: rclone-v1.69.1-linux-amd64/rclone.1
$
</code>

----

===== Running =====

Simply change to the directory that you unzipped Rclone to, and run the ''rclone'' executable:

<code lang=shell-session>
$ cd rclone-v1.69.1-linux-amd64
$ ./rclone version
rclone v1.69.1
- os/version: redhat 7.9 (64 bit)
- os/kernel: 3.10.0-1160.125.1.el7.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.24.0
- go/linking: static
- go/tags: none
$
</code>

You can copy the ''rclone'' file anywhere in your **HOME** or **NOBACKUP** area. If you have a directory for [[https://en.wikipedia.org/wiki/Man_page|man]] pages you can copy the ''rclone.1'' to that folder, but it is not essential - the ''README.txt'' and ''README.html'' files contain the same help information.

Support for the //use// of Rclone is outside of what we are able to provide - please consult the (extensive) Rclone documentation:

   * https://rclone.org/docs/

----

[[:advanced:software|Back to Advanced Software index]]