Load data from AWS S3 to Snowflake

Load the S3 bucket file(s) into a Snowflake table, You may specify the file format options and copy options appropriately.

Load data from Azure Storage to Snowflake

Load the Azure container blobs into a Snowflake table, You may specify the file format options and copy options appropriately.

Load data from Google storage to Snowflake

Load the google storage bucket files into a Snowflake table, You may specify the file format options and copy options appropriately. Please note that you will need to define the storage integration in Snowflake  and provide this in the Universal Task.

Load Internal stage file to Snowflake Table

Files that are available in Snowflake internal storage to be loaded into a Snowflake table using the pipe name and authentication using private public key files. The data can be loaded in Snowflake using patterns.

Copy from local server to Internal staging

Copies files from local Windows or Linux server to Snowflake internal staging area.

Unload Snowflake data to AWS S3

This feature helps to unload the data from a Snowflake table to a AWS S3 bucket, file format options and copy options can be provided appropriately.

Unload Snowflake data to Azure Storage

This feature helps to unload the data from a Snowflake table to an Azure container; file format options and copy options can be provided appropriately.

Unload Snowflake data to Google Storage

This feature helps to unload the data from a Snowflake table to a Google cloud Storage; file format options and copy options can be provided appropriately. Also, this would need an storage integration name from Snowflake.

Unload Snowflake data to Internal stage

Unloads the Snowflake table into an internal staging area in Snowflake.

Unload from internal stage to local server

This feature helps to copy the files in staging area to a local windows or a Linux server.

Execute a Snowflake command

Users can use this feature to execute snowflake commands; for example: Copy, Remove, Select, Delete, etc.

Snowflake Account

Provide the Snowflake user account; for example, sr14548.eu-central-1.

Snowflake Host URL

Specify your host information in the form of a URL; for example, sr14548.eu-central-1.snowflakecomputing.com.

Snowflake Login Name

Specify your Snowflake login name..

Log Level

Select a log level.

Select a Snowflake function

Select the required Snowflake function (if the required function not available, check the SQL task to invoke snowflake DB).

Snowflake Command

Provide a Snowflake command either in SQL or put / get commands etc.

Local file Name & Path

Provide the local file name that need to be copied to stage.

Stage Name

Provide the stage(internal) name in Snowflake.

Encrypted Private Key

Select only if you have a encrypted private key.

Stage File Name(s)

Provide the stage file names; if there are multiple files, separate by comma.

Pipe Name

Specify the fully-qualified name of the pipe to use to load the data.

Private Key File Path

Provide the private key file path to establish connection to Snowflake for data ingest.

Private Key Password

Provide the password for private if it is encrypted.

Snowflake Table Name

Provide the full path and the Snowflake table name where the table to be loaded.

Use AWS Credentials

Check this if you need to supply AWS access key credentials.

AWS Key ID & Secret Access Key

Provide the AWS secret Access Key (runtime user AWS Key id and secret key in the password section).

AWS Storage Integration

Provide the name of the AWS storage integration created in Snowflake.

Load using Pattern

If you need load the data using pattern, check this option.

S3 Bucket URL

Provide your S3 bucket URL s3://<your_s3_bucket>/data/.

Azure container File

Provide the Azure container file URL starting with azure://

Azure Storage Integration

Provide the Azure storage integration created in Snowflake.

GCP storage URL

Provide the bucket name and the file name; for example, gcs://mybucket/data/files.

Azure Sas Token

Provide the Azure Sas Token.

Existing File Format Name

Specify an existing named file format to use for loading data into the table.

Storage Integration

Provide the Snowflake storage integration details.

Pattern

Regular expression pattern string - specifying the file names and/or paths to match.

File Format Name

Specifies an existing named file format to use for loading data into the table.

Format Type

Specify the format type: CSV | JSON | AVRO | ORC | PARQUET | XML.

File Format Type Options

Specify the file format type options; for example, FIELD_DELIMITER = '|' if CSV type -- Refer to https://docs.snowflake.com/en/sql-reference/sql/copy-into-table.html

Copy Options

Include other copy options; for example, ON_ERROR = CONTINUE or FORCE=TRUE- Refer to https://docs.snowflake.com/en/sql-reference/sql/copy-into-table.html

Polling Interval (Secs)

Provide the Polling Interval time in Seconds, in the case of data ingestion from internal staging to Snowflake Table.

Number of times to Poll

Specify the number of times to poll; otherwise, default is set to 50, in the case of data ingestion from internal staging to Snowflake Table.

Universal Templates

User documentation for creating Universal Templates in the Universal Controller user interface.

Universal Tasks

User documentation for creating Universal Tasks in the Universal Controller user interface.