Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Panel
Table of Contents

Disclaimer

...

Warning

This is a major release and introduces breaking changes that might affect some users depending on their setup. Administrators are strongly advised to refer to Changelog for more information on the changes introduced in this release.

Overview

This integration provides the capability to perform data transfers between cloud-based storage services and local or distributed file systems. It also provides data storage management capabilities like listing, creating, or deleting data storage objects. 


Key Features

This integration is equipped with the following key features.

...

NameTypeDescriptionVersion Information
ActionChoiceAction to be performed.

Available actions:
  • Copy Objects (default)
  • Copy And Rename
  • Copy From URL
  • Move Objects
  • Move And Rename
  • List Objects
  • List Directories
  • List Objects With Details
  • List Objects In JSON Format
  • Create Object
  • Delete Object
  • Delete Empty Directory
  • Synchronize
Introduced in 1.0.0
Configuration FileScript

Rclone Configuration file, which contains all required parameters to connect to the Storage System, Source Storage and Target Storage. The filename should have the ".conf" suffix.

Example: rclone.conf

Introduced in 1.0.0
Storage SystemDynamic Choice FieldRemote or Local storage to execute the selected Action.
Execute the Dynamic Choice Field and retrieve all the available Storages from Configuration File.

Required when one of the Single storage actions is selected.
Introduced in 1.0.0
Source StorageDynamic Choice FieldRemote or Local source storage, where transfer Action is initiated.
Execute the Dynamic Choice Field and retrieve automatically all the available Storages from Configuration File.

Required when one of the Two storage actions is selected.
Introduced in 1.0.0
Target StorageDynamic Choice FieldRemote or Local target storage, where transferred data is stored.
Execute the Dynamic Choice Field and retrieve automatically all the available Storages from Configuration File.

Required when one of the Two storage actions is selected.
Introduced in 1.0.0
Storage CredentialsCredentialCredentials needed to connect to the selected Storage System.

Required when one of the Single storage actions is selected.

For Storage System of type local, the standard Agent's Credentials field should be provided.
Introduced in 1.0.0
Source CredentialsCredentialCredentials needed to connect to the selected Source Storage.

Required when one of the Two storage actions is selected. For Storage System of type local, the standard Agent's Credentials field should be provided.
Introduced in 1.0.0
Target CredentialsCredentialCredentials needed to connect to the selected Target Storage.

Required when one of the Two storage actions is selected.

For Storage System of type local, the standard Agent's Credentials field should be provided.
Introduced in 1.0.0
FilepathTextFile or directory path from where to retrieve data for the selected Action.

Required when one of the Single storage actions is selected.
Introduced in 1.0.0
Source FilepathTextFile or directory path from where to retrieve data for the selected Action.

Required when one of the Two storage actions is selected.
Introduced in 1.0.0
Target FilepathTextFile or directory path where transferred data will be stored for the selected Action.

Required when one of the Two storage actions is selected.
Introduced in 1.0.0
Update CredentialsCheckbox 

If the remote storage uses OAuth for its authorization, Rclone can be adjusted to refresh the respective token in its Configuration File during execution time. Enabling this field (and as soon as Rclone is adjusted to refresh tokens), will ensure that the runtime-updated information from Rclone, will get provisioned to the respective Credential Fields on the Universal Controller.

This field is used in conjunction with fields Controller URL, Controller Credentials, Refresh Storage Credentials.

The default setting is unchecked.

Introduced in 1.0.0
Controller URLTextController URL where Storage Credentials, Source Credentials, Target Credentials are stored.

Required when Update Credentials is checked.
Introduced in 1.0.0
Controller CredentialsCredentialsController user's credentials, used for logging and updating Storage Credentials, Source Credentials, Target Credentials. The Credentials definition should be as follows:
  • User as "Runtime User".
  • User Password as "Runtime Password".
Note: The provided user should have sufficient Credential permissions.

Required when Update Credentials is checked.
Introduced in 1.0.0

Refresh Storage Credentials

Anchor
Refresh Storage Credential
Refresh Storage Credential

ArrayThis Array field is available when Update Credentials is checked. Is used as a mapping table between the Configuration File locator that has been updated by Rclone during execution, and its corresponding Universal Controller Credential field.

Populate this array with the following format:
  • Configuration File Details: "<Storage/Source/Target Storage>"."<Configuration File's field to be refreshed by Rclone>"
  • Storage Credentials Details: "<Storage/Source/Target Credentials name>"."<Storage/Source/Target Credentials field to be updated>"

The result of this action is that the Configuration File's locator value, is copied to the provided Storage Credential's field. Each line of this Array is evaluated, and related mapping is performed.

Note: Refresh of Storage/Source/Target Credentials is performed via Controller's REST API. See Network Requirements.

Introduced in 1.0.0
Use FilterChoiceThe filter type that is applied on the Action.

Available options:
  • -- None – (default)
  • Include
  • Exclude

Optional for all, but Create Object Action.
Introduced in 1.0.0
FilterTextFilter as regular expression that is applied on the Action.

Required when Use Filter is checked.
Introduced in 1.0.0
Overwrite OptionsChoiceOptions for overwriting files in the Target Storage.

Available options:
  • Overwrite
  • Do Not Overwrite (default)
  • Append Timestamp In Filename

Optional when one of the Two storage actions is selected.
Introduced in 1.0.0
Recursion DepthIntegerRecursion depth that is applied. Default value 1 means that no recursion will be applied.

Note: Use this option with caution when one of the following Actions are selected, as this might result in doing actions on unnecessary files/cloud objects. For more information on Recursion Depth, you can refer to the official Rclone documentation:
  • Copy Objects
  • Copy And Rename
  • Move Objects
  • Move And Rename
  • Delete Object
  • Delete Empty Directory
  • Synchronize


The default value is 1.

Introduced in 1.0.0
Error On No File TransferCheckbox

When enabled and when no files are transferred a failure exit code with value equal to 21 is raised.

Optional when one of the Two storage actions is selected.


The default setting is unchecked.

Introduced in 1.0.0
Dry-RunCheckboxWhen enabled, performs a trial run with no permanent changes of the selected Action.

Note: It is recommended to execute a Dry-Run as a test prior to the final task configuration when the following Actions are used:
  • Copy Objects
  • Copy And Rename
  • Move Objects
  • Move And Rename
  • Delete Object
  • Delete Empty Directory
  • Synchronize


The default value is unchecked.

Introduced in 1.0.0
Additional OptionsTextSpace-separated Rclone options applied to the selected Action.Introduced in 1.0.0
Log FormatChoiceOption for STDERR logging format.

Available options:
  • Text (default)
  • JSON
Introduced in 1.0.0

...