SAP HANA Cloud - data lake

Disclaimer

Your use of this download is governed by Stonebranch's Terms of Use.

info

For general information on how integrations work, importing templates, and UAC documentation refer to Overview.

Version Information

Template Name	Extension Name	Version	Status
SAP HANA Cloud - data lake	ue-sap-datalake	1.0.0	Initial Version.

Refer to Changelog for version history information.

Overview

SAP HANA Cloud, data lake is a managed cloud service for storing and analyzing large volumes of structured and unstructured data. Its Files component stores data as files in file containers, and its Relational Engine provides SQL access to the data in the data lake.

This Universal Task allows customers to transfer, list and delete files in SAP HANA Cloud, data lake Files, and to execute SQL scripts against the data lake Relational Engine, straight from the Universal Controller.

Key Features

Feature	Description
File Transfer	Upload local files to, and download files from, a data lake file container, with control over existing files and source retention.
File Management	List and delete data lake files selected by a glob pattern and a recursion depth.
SQL Script Execution	Run multi-statement SQL scripts against the data lake Relational Engine.

Requirements

This integration requires a Universal Agent and a Python runtime to execute the Universal Task.

Area	Details
Python Version	Requires Python 3.11 or 3.14, tested with Agent bundled python distribution.
Universal Agent Compatibility	Compatible with Universal Agent for Windows x64 and version >= 7.8.0.0. Compatible with Universal Agent for Linux and version >= 7.8.0.0.
Universal Controller Compatibility	Universal Controller Version >= 7.8.0.0.
Network and Connectivity	Network connectivity to the SAP HANA Cloud, data lake Files REST endpoint and, for Execute SQL, to the Relational Engine endpoint is required. The allowed connections of the data lake instance must include the agent's IP address.
Client Certificate	A PEM client certificate and matching private key, readable by the agent user, are required for the file actions. Authentication to data lake Files uses mutual TLS: the certificate must be signed by a CA uploaded to the instance as a trust, and an authorization entry matching the certificate's subject must grant it a role.
SAP HANA Cloud, Data Lake Client	The SAP HANA Cloud, data lake client must be installed on the agent for Execute SQL. On Windows, the client does not include a TLS library, so the OpenSSL 3 libraries libssl-3-x64.dll and libcrypto-3-x64.dll (64-bit) must be placed in the bin64 folder of the client installation.

Supported Actions

There are 5 Top-Level actions controlled by the Action Field:

• Upload File: Uploads local files to a data lake directory.
• Download File: Downloads data lake files to a local directory.
• List Files: Lists data lake files under a remote directory.
• Delete Files: Deletes data lake files under a remote directory.
• Execute SQL: Runs an SQL script against the Relational Engine.

Action: Upload File

Uploads every local file matching the File Pattern from the Local Directory into the Remote Directory. Operation determines whether the local source files are kept or removed, and Write Option determines how a file that already exists at the destination is handled.

Action: Download File

Downloads every data lake file matching the File Pattern from the Remote Directory into the Local Directory. Operation determines whether the data lake source files are kept or removed, and Write Option determines how a file that already exists at the destination is handled.

Action: List Files

Lists every data lake file matching the File Pattern under the Remote Directory. No file is changed.

Action: Delete Files

Deletes every data lake file matching the File Pattern under the Remote Directory. The deletion continues past a failed file, and the task fails with a transfer error when any file could not be deleted.

Action: Execute SQL

Runs the SQL Script against the Relational Engine one statement at a time, using the user and password of the SQL Credential. The script stops at the first failed statement. The statements run in one transaction that is committed after the last statement, and DDL statements are committed implicitly by the Relational Engine.

Action Output

EXTENSION

The extension output provides the following information:

• exit_code, status_description: General info regarding the task execution. For more information refer to exit codes table.
• invocation.fields: The task configuration used for this task execution.
• result.transferred, result.skipped: The data lake or local paths transferred or skipped by Upload File and Download File, along with result.transferred_count and result.skipped_count.
• result.files: The fully qualified paths of the files found by List Files, along with result.file_count.
• result.deleted: The paths of the files removed by Delete Files, along with result.deleted_count.
• result.statement_count, result.committed: The number of statements in the script run by Execute SQL, and whether the final commit succeeded.
• result.statements: One entry per statement run by Execute SQL, with index, status, rows_affected, columns, rows, truncated and duration_ms. A statement returns at most 100 rows, a limit that can be changed with UE_EXTENSION_OUTPUT_MAX_SQL_ROWS, and truncated is true when more rows were available. A string value longer than 4096 characters is cut and marked. The statement text is not part of the output.
• result.sqlcode, result.failed_statement, result.ddl_may_have_committed: Present when Execute SQL fails. They identify the failed statement and state whether earlier work may have been saved, because DDL statements are committed implicitly.
• result.errors: List of errors that might have occurred during execution.

Examples:

Successful execution

{
  "exit_code": 0,
  "status_description": "Uploaded 2 file(s), skipped 0",
  "invocation": {
    "extension": "ue-sap-datalake",
    "version": "1.0.0",
    "fields": {
      "action": "Upload File",
      "base_url": "https://ed306ac8-7da9-4d85-b947-248a340f0a17.files.hdl.prod-eu10.hanacloud.ondemand.com",
      "file_container": "ed306ac8-7da9-4d85-b947-248a340f0a17",
      "client_cert": "/opt/universal/certs/hdlfs-client.crt",
      "client_key": "/opt/universal/certs/hdlfs-client.key",
      "key_passphrase": null,
      "local_path": "/data/outbound/sales",
      "remote_path": "/landing/sales",
      "file_pattern": "*.csv",
      "recursion_depth": 2,
      "operation": "Copy",
      "write_option": "Replace",
      "sql_endpoint": null,
      "sql_credential": null,
      "sql_script": null,
      "hdl_client_home": null,
      "verify_ssl": null
    }
  },
  "result": {
    "transferred": [
      "/landing/sales/orders_20261006.csv",
      "/landing/sales/2026/orders_20261007.csv"
    ],
    "skipped": [],
    "transferred_count": 2,
    "skipped_count": 0
  }
}

Failed execution

{
  "exit_code": 20,
  "status_description": "Data Validation Error: '/data/outbound/sales' is not a directory.",
  "invocation": {
    "extension": "ue-sap-datalake",
    "version": "1.0.0",
    "fields": {
      "action": "Upload File",
      "base_url": "https://ed306ac8-7da9-4d85-b947-248a340f0a17.files.hdl.prod-eu10.hanacloud.ondemand.com",
      "file_container": "ed306ac8-7da9-4d85-b947-248a340f0a17",
      "client_cert": "/opt/universal/certs/hdlfs-client.crt",
      "client_key": "/opt/universal/certs/hdlfs-client.key",
      "key_passphrase": null,
      "local_path": "/data/outbound/sales",
      "remote_path": "/landing/sales",
      "file_pattern": "*.csv",
      "recursion_depth": 2,
      "operation": "Copy",
      "write_option": "Replace",
      "sql_endpoint": null,
      "sql_credential": null,
      "sql_script": null,
      "hdl_client_home": null,
      "verify_ssl": null
    }
  },
  "result": {
    "errors": ["Data Validation Error: '/data/outbound/sales' is not a directory."]
  }
}

Successful execution (Execute SQL)

{
  "exit_code": 0,
  "status_description": "Executed 2 statement(s)",
  "invocation": {
    "extension": "ue-sap-datalake",
    "version": "1.0.0",
    "fields": {
      "action": "Execute SQL",
      "base_url": null,
      "file_container": null,
      "client_cert": null,
      "client_key": null,
      "key_passphrase": null,
      "local_path": null,
      "remote_path": null,
      "file_pattern": null,
      "recursion_depth": null,
      "operation": null,
      "write_option": null,
      "sql_endpoint": "ed306ac8-7da9-4d85-b947-248a340f0a17.iq.hdl.prod-eu10.hanacloud.ondemand.com:443",
      "sql_credential": {
        "user": "HDLADMIN",
        "password": "****",
        "token": "****",
        "key_location": null,
        "passphrase": "****"
      },
      "sql_script": "/opt/universal/tmp/orders_daily.sql",
      "hdl_client_home": "/opt/sap/hdlclient",
      "verify_ssl": true
    }
  },
  "result": {
    "statement_count": 2,
    "committed": true,
    "statements": [
      {
        "index": 1,
        "status": "ok",
        "rows_affected": 15230,
        "columns": [],
        "rows": [],
        "truncated": false,
        "duration_ms": 2310
      },
      {
        "index": 2,
        "status": "ok",
        "rows_affected": 1,
        "columns": ["ORDER_COUNT"],
        "rows": [{ "ORDER_COUNT": 15230 }],
        "truncated": false,
        "duration_ms": 87
      }
    ]
  }
}

STDOUT

The file actions print the number of files processed, followed by a tree of those files under the directory used. Files moved by Upload File or Download File are marked (moved) on the count line. Execute SQL prints one line per statement with its index, status, and either the affected row count and duration or the error. The statement text is never printed.

The output reaches the Universal Controller only when Automatic Output Retrieval is enabled on the task.

Examples:

Uploaded 2 file(s).
/landing/sales
|-- orders_20261006.csv
`-- 2026
    `-- orders_20261007.csv

[1] ok: 15230 row(s) in 2310 ms
[2] ok: 1 row(s) in 87 ms

Configuration Examples

Example: Upload File, copy, replace existing files, recursive

Uploads every file matching */sample_*.csv up to two directory levels below /tmp/ue_sap_datalake/outbound on the agent into /uc1_example/landing of the file container, keeping the subdirectories. The data lake endpoint and client certificate are provided in Base URL, File Container, Client Certificate File and Client Private Key File. Operation is "Copy", so the local files remain, and Write Option is "Replace", so existing data lake files with the same name are overwritten. The task ends successfully once every matching file is uploaded, and the Extension Output lists the uploaded files in result.transferred.

Example: Execute SQL, multi-statement script, SSL verification

Runs a multi-statement SQL script against the Relational Engine at the SQL Endpoint, logging in with the user and password of the SQL Credential. HDL Client Home points to the data lake client installation on the agent, and Verify SSL is checked so the server certificate is validated. The task ends successfully after the last statement completes, and the per-statement results are returned in result.statements.

Input Fields

Name	Type	Description	Version Information
Action	Choice	The operation to perform against SAP HANA Cloud, data lake. Available options: Upload File (default) - uploads local files to the data lake. Download File - downloads data lake files to the agent. List Files - lists data lake files. Delete Files - deletes data lake files. Execute SQL - runs an SQL script against the Relational Engine.	Introduced in 1.0.0
Base URL	Text	The REST endpoint of the data lake Files instance, for example https://<instance-id>.files.hdl.prod-<region>.hanacloud.ondemand.com. Available and required if Action is "Upload File", "Download File", "List Files" or "Delete Files".	Introduced in 1.0.0
File Container	Text	The data lake instance ID, which identifies the file container. Sent as the x-sap-filecontainer header. Available and required if Action is "Upload File", "Download File", "List Files" or "Delete Files".	Introduced in 1.0.0
Client Certificate File	Text	The path on the agent to the PEM client certificate used for mutual TLS. The file must be readable by the agent user. Available and required if Action is "Upload File", "Download File", "List Files" or "Delete Files".	Introduced in 1.0.0
Client Private Key File	Text	The path on the agent to the PEM private key matching the client certificate. The file must be readable by the agent user. Available and required if Action is "Upload File", "Download File", "List Files" or "Delete Files".	Introduced in 1.0.0
Private Key Passphrase	Credential	The Credential whose Runtime Password holds the private key's passphrase. Left empty when the key is not encrypted. Available if Action is "Upload File", "Download File", "List Files" or "Delete Files".	Introduced in 1.0.0
Local Directory	Text	The directory on the agent that is the upload source or the download destination. The directory must already exist. Available and required if Action is "Upload File" or "Download File".	Introduced in 1.0.0
Remote Directory	Text	The directory inside the file container that is the upload destination, the download source, or the location searched for files to list or delete. Available and required if Action is "Upload File", "Download File", "List Files" or "Delete Files".	Introduced in 1.0.0
File Pattern	Text	The Unix glob matched against each file's path relative to the directory, not just its name, on both the local and the data lake side. For example, sample_*.csv does not match emea/sample_sales.csv, while */sample_*.csv does. * crosses /, so *.csv matches CSV files at every level. Matching is case-sensitive, except on the local side of a Windows agent. Defaults to *, which matches every file. Available if Action is "Upload File", "Download File", "List Files" or "Delete Files".	Introduced in 1.0.0
Recursion Depth	Integer	The number of directory levels to include. 1 means no recursion. A pattern containing / requires a depth of at least its number of segments. Defaults to 1. Available if Action is "Upload File", "Download File", "List Files" or "Delete Files".	Introduced in 1.0.0
Operation	Choice	Whether the source file is kept or removed after a successful transfer. Available options: Copy (default) - leaves the source file in place. Move - deletes each source file once it has transferred successfully. Available if Action is "Upload File" or "Download File".	Introduced in 1.0.0
Write Option	Choice	How a file that already exists at the destination is handled. Available options: Replace (default) - overwrites the existing file. Do Not Overwrite - skips the file. Timestamp - writes the file under a timestamped name when the file already exists. Always Timestamp - always writes the file under a timestamped name. Rename - writes the file under a new name when the file already exists. Available if Action is "Upload File" or "Download File".	Introduced in 1.0.0
SQL Endpoint	Text	The Relational Engine endpoint as host:port, for example <instance-id>.iq.hdl.prod-<region>.hanacloud.ondemand.com:443. Available and required if Action is "Execute SQL".	Introduced in 1.0.0
SQL Credential	Credential	The UAC Credential whose Runtime User and Runtime Password log in to the Relational Engine. Available and required if Action is "Execute SQL".	Introduced in 1.0.0
SQL Script	Script	The SQL statements to run, separated by semicolons, in order. The first failed statement stops the script. Compound statements (BEGIN ... END) are not supported. Available and required if Action is "Execute SQL".	Introduced in 1.0.0
HDL Client Home	Text	The directory where the SAP HANA Cloud, data lake client is installed on the agent. Available and required if Action is "Execute SQL".	Introduced in 1.0.0
Verify SSL	Checkbox	Check to validate the Relational Engine's server certificate against the CA bundle named by UE_SSL_CERT_FILE, or against the default trust store when the variable is not set. Available if Action is "Execute SQL".	Introduced in 1.0.0

Environment Variables

Environment Variables can be set from the Environment Variables task definition table.

Environment Variable Name	Description	Version Information
UE_HTTP_TIMEOUT	The timeout in seconds of each request to data lake Files. Defaults to 300. A value that is not a positive integer is replaced by the default.	Introduced in 1.0.0
UE_HTTP_RETRIES	The number of retries when a connection to data lake Files cannot be established. Defaults to 3. Set to 0 to disable retries.	Introduced in 1.0.0
UE_HTTP_INTERVAL	The number of seconds between connection attempts to data lake Files. Defaults to 5.	Introduced in 1.0.0
UE_SSL_CERT_FILE	The path to a CA bundle used to validate the server certificate of data lake Files and, when Verify SSL is checked, of the Relational Engine. Defaults to the system trust store. The task fails when the path is set but is not a valid CA bundle file.	Introduced in 1.0.0
HTTPS_PROXY	The proxy used for connections to data lake Files over HTTPS. The lowercase https_proxy is also read. Not applied to Execute SQL.	Introduced in 1.0.0
HTTP_PROXY	The proxy used for connections to data lake Files over HTTP. The lowercase http_proxy is also read. Not applied to Execute SQL.	Introduced in 1.0.0
ALL_PROXY	The proxy used for connections to data lake Files when no protocol-specific proxy variable applies. The lowercase all_proxy is also read. Not applied to Execute SQL.	Introduced in 1.0.0
NO_PROXY	The comma-separated list of hosts that bypass the proxy when connecting to data lake Files. The lowercase no_proxy is also read.	Introduced in 1.0.0
UE_EXTENSION_OUTPUT_MAX_SQL_ROWS	The maximum number of rows returned in the Extension Output for each Execute SQL statement. Defaults to 100. A value that is not a positive integer is replaced by the default.	Introduced in 1.0.0

Cancellation and Rerun

• In case of cancellation, Upload File and Download File stop at the next chunk of the file in progress, and Delete Files stops before the next file. The task ends with exit code 40. Files already uploaded, downloaded or deleted remain so, and with "Move" the sources of files already transferred stay removed. The paths processed are returned in result.transferred, result.skipped and result.errors, or in result.deleted and result.errors for Delete Files.
• In case of cancellation, Execute SQL asks the Relational Engine to stop the running statement, rolls back the open transaction and ends with exit code 40. The final commit does not run. Statements that ran before the cancellation are listed in result.statements, and result.ddl_may_have_committed states whether earlier work may have been saved, because DDL statements are committed implicitly by the Relational Engine.
• In case of rerun, the task starts a new execution from the beginning. File actions are evaluated again against the current contents of both sides, with Write Option determining how files that already exist at the destination are handled, and Execute SQL runs the whole script again from the first statement.

Importable Configuration Examples

This integration provides importable configuration examples along with their dependencies, grouped as Use Cases to better describe end to end capabilities.

warning

These examples aid in allowing task authors to get more familiar with the configuration of tasks and related Use Cases. Such tasks should be imported in a Test system and should not be used directly in production.

Initial Preparation Steps

• STEP 1: Go to Stonebranch Integration Hub and download the "SAP HANA Cloud - data lake" and "UAC Utility: Email" integrations. Extract the downloaded archives in a local directory.
• STEP 2: Locate and import the above integrations to the target Universal Controller. For more information refer to the How To section in this document.
• STEP 3: Inside the directory named "configuration_examples" of the SAP HANA Cloud - data lake archive you will find a list of definition zip files. Upload them one by one respecting the order presented below, by using the "Upload" functionality of Universal Controller:
  • 1_variables.zip
  • 2_credentials.zip
  • 3_tasks.zip
  • 4_workflows.zip
• STEP 4: Update the uploaded UAC Credential entity(ies) with the required credentials. Review each credential's description for guidance on what to provide.
• STEP 5: Update the UAC global variables introduced with the *_variables.zip file. Their name is prefixed with the extension name using underscores instead of hyphens. Review the descriptions of the variables as they include information on how they should be populated.
• STEP 6: If applicable, create an OMS record on the Universal Controller. Ensure the address matches the one assigned to the relevant global variable from the previous step. For more information refer to Creating OMS Server Records.
• STEP 7: If applicable, create an Agent Cluster on the Universal Controller. Ensure the name matches the one assigned to the relevant global variable from the previous step. For more information refer to Creating an Agent Cluster.

tip

• The order indicated above ensures that the dependencies of the imported entities need to be uploaded first.
• All imported entities are prefixed with the Use Case number (e.g. UC1, UC2, UC3) they belong to.

How to "Upload" Definition Files to a Universal Controller

The "Upload" functionality of Universal Controller allows Users to import definitions exported with the "Download" functionality.

Login to Universal Controller and:

• STEP 1: Click "Tasks" → "All Tasks"
• STEP 2: Right click on the top of the column named "Name"
• STEP 3: Click "Upload..."

In the pop-up "Upload..." dialogue:

• STEP 1: Click "Choose File".
• STEP 2: Select the appropriate zip definition file and click "Upload".
• STEP 3: Observe the Console for possible errors.

Use Case 1: Deliver Agent Extracts to an SAP HANA Cloud, data lake Landing Zone

Description

This Use Case creates sample CSV extracts on an agent, copies them into a landing directory of a data lake file container, and lists the landing directory to confirm what was delivered. When the upload fails, an email notifies the recipients.

The workflow "UC1: SAP HANA Cloud - data lake - Deliver Agent Extracts to the Data Lake" is composed of the following components:

1. UC1: SAP HANA Cloud - data lake - Create Sample Extracts: A Linux/Unix Task that writes emea/sample_sales.csv and amer/sample_sales.csv under the agent directory set by ue_sap_datalake_local_path. It stands in for the job that produces real extracts.
2. UC1: SAP HANA Cloud - data lake - Upload Extracts: Uploads every file matching */sample_*.csv, with a Recursion Depth of 2, into the landing directory set by ue_sap_datalake_remote_path, keeping the subdirectories. Operation is "Copy", so the agent files are kept, and Write Option is "Replace", so a rerun overwrites the files already delivered.
3. UC1: SAP HANA Cloud - data lake - List Landed Files: Runs when the upload succeeds and lists the landing directory with the same pattern and depth.
4. UC1: SAP HANA Cloud - data lake - Send Email for Failure: Runs when the upload fails and emails the recipients set by ue_sap_datalake_email_to, using the Email integration with Microsoft App Only authentication.

How to Run

Execution Steps

1. Ensure all Initial Preparation Steps have been completed successfully.
2. Ensure the agent set by ue_sap_datalake_agent is a Linux/Unix agent, and that the client certificate and private key set by ue_sap_datalake_client_cert and ue_sap_datalake_client_key are trusted and authorized by the data lake Files instance.
3. Launch the workflow task and monitor the execution in Universal Controller.

Expected Results

• emea/sample_sales.csv and amer/sample_sales.csv are uploaded into the landing directory, keeping their subdirectories.
• The listing task returns the delivered files in result.files of its Extension Output.
• The workflow completes in Success state when the upload and the listing succeed.
• To see the failure path, change the File Pattern of the upload task to *.json. No file matches, the upload ends with exit code 1, and the email is sent.

Exit Codes

Exit Code	Status	Description
0	Success	Successful Execution.
1	Failure	Generic Error. Raised when not falling into the other Error Codes, for example when no file matches the File Pattern or an SQL statement fails.
2	Failure	Authentication error. The data lake rejected the client certificate, or the Relational Engine rejected the SQL Credential.
3	Failure	Authorization error. The client certificate or the SQL user lacks the required privilege.
10	Failure	Connection error. The data lake Files or Relational Engine endpoint could not be reached, or the service is unavailable.
20	Failure	Input fields validation error.
21	Failure	Not found. The remote directory or file does not exist.
30	Failure	Transfer failed. One or more files could not be uploaded, downloaded or deleted, while others may have succeeded or the files failed for different reasons. When nothing was transferred and every file failed for the same reason, the exit code of that reason is returned instead.
40	Failure	Cancelled. The task instance was cancelled during execution.

Document References

Document Link	Description
SAP HANA Cloud, Data Lake Files REST API Reference	Describes the REST API of data lake Files used by the file actions.
SAP HANA Cloud, Data Lake Files Getting Started	Describes how to provision a data lake Files instance and create file containers.
SAP HANA Cloud, Data Lake Files Client Certificates	Describes how to configure the client certificates used for mutual TLS authentication.
SAP HANA Cloud, Data Lake Client Interfaces Reference	Describes how to install the data lake client required by Execute SQL.
SAP HANA Cloud, Data Lake Relational Engine SQL Statements Reference	Describes the syntax of the SQL statements that can be used in the SQL Script of Execute SQL.

Changelog

ue-sap-datalake-1.0.0 (2026-10-20)

Initial Version