Version: 3.3

Configuration JSON

The Configuration JSON defines the Rclone remote configuration for the selected cloud provider and object storage service (Type). It provides the connection parameters and credential variables that UDMG requires to authenticate and connect to a cloud provider.

This configuration must be provided as a JSON object compatible with the format used by Rclone. The Configuration JSON:

Identifies the target remote cloud storage provider
Specifies the required connection parameters
Uses credential placeholders instead of hardcoded secrets

Credentials

Authentication credentials for the cloud provider are configured at the Pipeline level. Any Pipeline that uses a Remote Cloud Storage Endpoint as a Source Endpoint or Destination Endpoint uses a Key Pair credential for authentication.

From the Remote Cloud Storage Endpoint perspective, the Configuration JSON only references that credential using the following placeholders:

{{ pipeline_credentials.key }}, which maps to the Key Pair Secret Key field.
{{ pipeline_credentials.value }}, which maps to the Key Pair Secret Value field.

When connecting to the cloud provider defined for the Remote Cloud Storage Endpoint at runtime, UDMG injects the associated Pipeline credential values.

Depending on the selected cloud provider, the Key Pair Secret Key and Secret Value fields may represent different authentication values. For details for each provider, see the Credentials Mapping section in the corresponding provider section.

warning

Do not include credential values directly in the Configuration JSON. Use the specified placeholders to ensure credentials are stored securely, injected at runtime, and managed centrally.

Required JSON Structure and Parameters

The Configuration JSON must follow the required JSON structure and include a type property that identifies the cloud provider, along with the provider-specific parameters required by Rclone.

The following sections describe the important JSON keys for each Remote Cloud Storage Endpoint type. Additional parameters (keys) can be added based your specific cloud provider and transfer needs.

AWS S3

AWS S3 is an object storage service provided by Amazon Web Services. In UDMG, it is used to connect to an S3 bucket.

The Configuration JSON must define the connection properties required for the selected S3 provider and reference credentials using placeholders. These values are resolved at runtime from the Key Pair Credential associated with the Pipeline.

info

For Rclone official documentation, refer to Amazon S3 Storage Providers.

JSON Properties

Property	Description	Expected Value(s)	Required
`type`	Identifies the Rclone backend.	`"s3"`	Yes
`provider`	Identifies the S3 provider.	`"AWS"`	Yes
`bucket`	Identifies the S3 bucket (container for objects stored). It must be unique and already exist. The `Cloud Base Storage Path` is appended as a suffix.	User-defined (`"udmg-sales"`)	Yes
`access_key_id`	The access key identifier.	`"{{ pipeline_credentials.key }}"`	Yes
`secret_access_key`	The secret access key.	`"{{ pipeline_credentials.value }}"`	Yes
`region`	The AWS region associated with the bucket.	User-defined (`"us-east-1"`)	Yes

Credentials Mapping

Property in Configuration JSON	Corresponding Field in Key Pair Credential	Description	Example
`access_key_id`	Secret Key	The AWS access key ID.	`AKIAIOSFODNN7EXAMPLE`
`secret_access_key`	Secret Value	The AWS secret access key.	`wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY`

Example

{
    "type": "s3",
    "provider": "AWS",
    "bucket": "udmg-sales",
    "access_key_id": "{{ pipeline_credentials.key }}",
    "secret_access_key": "{{ pipeline_credentials.value }}",
    "region": "us-east-1"
}

Google Cloud Storage

Google Cloud Storage is an object storage service provided by Google Cloud. In UDMG, it is used to connect to a Google Cloud Storage bucket.

The Configuration JSON must define the connection properties required for Google Cloud Storage and reference credentials using placeholders. These values are resolved at runtime from the Key Pair Credential associated with the Pipeline.

info

For Rclone official documentation, refer to Google Cloud Storage.

Service Account Credentials

Connecting to Google Cloud Storage requires a Google Service Account. Service accounts behave like standard user principals in Google Cloud Storage access control, so their permissions can be scoped to specific buckets or operations (for example, read-only access).

To obtain service account credentials:

Open the Service Accounts section of the Google Cloud Console.
Create or select a service account and assign the necessary bucket permissions.
Create a key for the service account. Google downloads a JSON credentials file (for example, GCS_PROJECT_ID_a7a371189f61.json) to your machine.

The entire content of that JSON file must be stored as the Secret Value in the Key Pair credential associated with the Pipeline. The Secret Key field is not used for Google Cloud Storage and is ignored.

The service account JSON file has the following structure:

{
  "type": "service_account",
  "project_id": "GCS_PROJECT_ID",
  "private_key_id": "a7a371189f61...",
  "private_key": "-----BEGIN PRIVATE KEY-----...",
  "client_email": "...",
  "client_id": "...",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/...",
  "universe_domain": "googleapis.com"
}

Once the credentials are stored in a Key Pair credential, they are referenced in the Configuration JSON using {{ pipeline_credentials.value }}. UDMG injects the credential value at runtime when connecting to Google Cloud Storage.

JSON Properties

The bucket and project properties are UDMG-specific fields that identify the target bucket and project. They are not present in a standard Rclone INI configuration and must be added manually.

Property	Description	Format	Required
`type`	Identifies the Rclone backend.	Must be `"google cloud storage"`	Yes
`project`	Set to the `project_id` value from the service account JSON file.	It is user-defined, for example, `"GCS_PROJECT_ID"`	Yes
`bucket`	The name of the Google Cloud Storage bucket. The Cloud Storage Base Path configured on the Endpoint is appended as a path suffix. The bucket must already exist.	It is user-defined, for example, `"my-gcs-bucket"`	Yes
`service_account_credentials`	The service account credential placeholder. UDMG resolves this value from the Key Pair credential at runtime.	Must be: `"{{ pipeline_credentials.value }}"`	Yes
`bucket_policy_only`	Must match the bucket's access control configuration in Google Cloud Storage. Set to `true` if the bucket has Uniform bucket-level access enabled; leave at the default `false` for buckets using fine-grained access control. A mismatch causes rclone ACL operation failures.	`true` or `false`	No. Defaults to `false`.

Credentials Mapping

Property in Configuration JSON Corresponding Field in Key Pair Credential Represents Example

Property in Configuration JSON	Corresponding Field in Key Pair Credential	Represents	Example
`service_account_credentials`	Secret Value	The full content of the Google Cloud service account JSON credentials file.	`{ "type": "service_account", "project_id": "GCS_PROJECT_ID", "private_key_id": "a7a371189f61...", "private_key": "-----BEGIN PRIVATE KEY-----...", "client_email": "...", "client_id": "...", "auth_uri": "https://accounts.google.com/o/oauth2/auth", "token_uri": "https://oauth2.googleapis.com/token", "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/...", "universe_domain": "googleapis.com" }`

service_account_credentials

Secret Value

The full content of the Google Cloud service account JSON credentials file.

{
    "type": "service_account",
    "project_id": "GCS_PROJECT_ID",
    "private_key_id": "a7a371189f61...",
    "private_key": "-----BEGIN PRIVATE KEY-----...",
    "client_email": "...",
    "client_id": "...",
    "auth_uri": "https://accounts.google.com/o/oauth2/auth",
    "token_uri": "https://oauth2.googleapis.com/token",
    "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
    "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/...",
    "universe_domain": "googleapis.com"
}

info

The Secret Key field in the Key Pair credential is not used for Google Cloud Storage and is ignored at runtime.

Example

{
    "type": "google cloud storage",
    "project": "GCS_PROJECT_ID",
    "bucket": "my-gcs-bucket",
    "service_account_credentials": "{{ pipeline_credentials.value }}",
    "bucket_policy_only": true
}

Converting from a Rclone Configuration

If you already use Rclone with Google Cloud Storage (for example, with the Inter-Cloud Data Transfer UAC extension), you can convert an existing Rclone INI configuration to the JSON format required by UDMG.

Given a Rclone INI section:

[google_cloud_storage_target]
type = google cloud storage
service_account_file = /home/stonebranch/GCS_PROJECT_ID_a7a371189f61.json
bucket_policy_only = true
project_number = 623724603950
location = Europe-west-4

Follow these steps to produce a valid UDMG Configuration JSON:

Remove the section header. Delete the [google_cloud_storage_target] line.
Replace service_account_file with service_account_credentials. Store the full content of the service account JSON file in a Key Pair credential Secret Value, then reference it using {{ pipeline_credentials.value }}. This keeps the credentials encrypted in the credentials store. Alternatively, the JSON content can be pasted directly as the value of service_account_credentials, but inline values are stored as plaintext in the Endpoint configuration.
Add project. Use the project_id value from the service account JSON file.
Add bucket. Specify the name of the Google Cloud Storage bucket.
Include any additional Rclone parameters (location, project_number, and so on) as-is.

The resulting Configuration JSON:

{
    "type": "google cloud storage",
    "service_account_credentials": "{{ pipeline_credentials.value }}",
    "project": "GCS_PROJECT_ID",
    "bucket": "my-gcs-bucket",
    "bucket_policy_only": true,
    "project_number": "623724603950",
    "location": "Europe-west-4"
}

Microsoft Azure Blob

Microsoft Azure Blob is an object storage service provided by Microsoft Azure. In UDMG, it is used to connect to an Azure Blob container.

The Configuration JSON must define the connection properties required for Microsoft Azure Blob and reference credentials using placeholders. These values are resolved at runtime from the Key Pair Credential associated with the Pipeline.

info

For Rclone official documentation, refer to Microsoft Azure Blob Storage.

JSON Properties

Property	Description	Expected Value(s)	Required
`type`	Identifies the Rclone backend.	`"azureblob"`	Yes
`account`	The Azure storage account name.	User-defined (`"udmgblob"`)	Yes
`key`	The Azure storage account key.	`"{{ pipeline_credentials.value }}"`	Yes
`container`	The Azure Blob container name.	User-defined (`"udmgcontainer"`)	No

Credentials Mapping

Property in Configuration JSON	Corresponding Field in Key Pair Credential	Represents	Example
`key`	Secret Value	The Azure storage account key.	`qwerty1234567890examplekey==`

Example

{
    "type": "azureblob",
    "account": "udmgblob",
    "key": "{{ pipeline_credentials.value }}",
    "container": "udmgcontainer"
}

Microsoft Azure File Storage

Microsoft Azure File Storage is a file storage service provided by Microsoft Azure. In UDMG, it is used to connect to an Azure file share.

The Configuration JSON must define the connection properties required for Microsoft Azure File Storage and reference credentials using placeholders. These values are resolved at runtime from the Key Pair Credential associated with the Pipeline.

info

For Rclone official documentation, refer to Microsoft Azure Files.

For detailed Azure File Storage-specific file transfer logic, see Cloud Storage - Azure File Exception.

JSON Properties

Property	Description	Expected Value(s)	Required
`type`	Identifies the Rclone backend.	`"azurefiles"`	Yes
`account`	The Azure storage account name.	User-defined (`"udmgfiles"`)	Yes
`key`	The Azure storage account key.	`"{{ pipeline_credentials.value }}"`	Yes
`share`	The Azure file share name.	User-defined (`"udmgfileshare"`)	No

Credentials Mapping

Property in Configuration JSON	Corresponding Field in Key Pair Credential	Represents	Example
`key`	Secret Value	The Azure storage account key.	`qwerty1234567890examplekey==`

Example

{
  "type": "azurefiles",
  "account": "udmgfiles",
  "key": "{{ pipeline_credentials.value }}",
  "share": "udmgfileshare"
}

How to Create the Configuration

Option 1: Generate from Rclone

This option may be useful if you already use Rclone and have an existing configuration for the selected cloud provider.

Rclone is a command-line tool that runs outside of UDMG. When you create a remote with Rclone, the configuration is stored in an INI-style format. UDMG does not use that format directly. Instead, the configuration must be converted into JSON before it is pasted into the Configuration JSON field.

To generate the JSON, follow these steps:

Run rclone config dump to export all configured remotes in JSON format.
Locate the JSON object corresponding to the remote you want to use.
Copy only the inner JSON object (excluding the remote name and outer structure) into the Configuration JSON field.
Replace any credential values with the required placeholders.
Verify that the resulting JSON complies with the Required JSON Structure and Parameters.

Option 2: Create Manually

You can write the Configuration JSON directly instead of generating it from an existing Rclone configuration.

When using this approach, ensure that the JSON structure, property names, and expected values comply with the Required JSON Structure and Parameters.

Credentials​

Required JSON Structure and Parameters​

AWS S3​

JSON Properties​

Credentials Mapping​

Example​

Google Cloud Storage​

Service Account Credentials​

JSON Properties​

Credentials Mapping​

Example​

Converting from a Rclone Configuration​

Microsoft Azure Blob​

JSON Properties​

Credentials Mapping​

Example​

Microsoft Azure File Storage​

JSON Properties​

Credentials Mapping​

Example​

How to Create the Configuration​

Option 1: Generate from Rclone​

Option 2: Create Manually​

Credentials

Required JSON Structure and Parameters

AWS S3

JSON Properties

Credentials Mapping

Example

Google Cloud Storage

Service Account Credentials

JSON Properties

Credentials Mapping

Example

Converting from a Rclone Configuration

Microsoft Azure Blob

JSON Properties

Credentials Mapping

Example

Microsoft Azure File Storage

JSON Properties

Credentials Mapping

Example

How to Create the Configuration

Option 1: Generate from Rclone

Option 2: Create Manually