Your use of this download is governed by Stonebranch’s Terms of Use, which are available at https://www.stonebranch.com/integration-hub/Terms-and-Privacy/Terms-of-Use/
This Integration allows Stonebranch users to schedule, trigger, and monitor the Azure Data Factory pipeline process directly from Universal Controller.
This integration uses Python modules azure-mgmt-resource and azure-mgmt-datafactory to make REST-API calls to Azure Data Factory.
This integration will use the Azure Tenant id , Subscription id , client id ,client secret, Resource group, and location for authenticating the REST-API calls to Azure Data Factory.
User can perform the following Azure Data Factory operations:
Run a Pipeline.
Get information on a Pipeline.
List all Pipelines.
Cancel Pipeline run.
List factory by resource group.
Also, with respect to Azure Data Factory triggers, users can perform the following operations from UAC:
Start Trigger.
Stop Trigger.
List Trigger by Factory.
UAC also can restart a failed pipeline either from the failed step or from any activity name in the failed pipeline.
This integration requires a Universal Agent and a Python runtime to execute the Universal Task against an Azure Data Factory.
Requires Python 3.4 or higher. Tested with the Universal Agent bundled Python distribution.
Python modules required:
azure-mgmt-resource
azure-mgmt-datafactory
Either:
Universal Agent for Windows x64 Version 6.9 and later with Python options installed.
Universal Agent for Linux Version 6.9 and later with Python options installed.
Universal Controller Version 6.9.0.0 and later.
This Universal Task has been tested with the Azure Data Factory Version 2.
This integration uses the Python modules Azure resource Management Module and Azure Data Factory management to make REST-API calls with Azure Data Factory.
Use Azure App Registration services to create an app, client ID, and client and associate it with Data Factory to provide appropriate roles (for example, Contributor).
Feature | Description |
Run a Pipeline | Execute a pipeline defined in Azure Data Factory. Schedule, trigger, and monitor the execution of a pipeline. |
Get Information on a Pipeline | Get a information about the pipeline name . |
List all Pipelines | List all pipelines that belongs to a Data Factory. |
Cancel a Pipeline Run | Abort a Pipeline execution from UAC by providing the run ID for the Pipeline execution. |
List Factory by resource group | List the available factory name that belongs to a resource group. |
Start a Trigger | Start a trigger that will manage pipeline execution schedule in the Azure Data Factory. |
Stop a Trigger | Stop a trigger that is in the Azure Data Factory. |
List Trigger by Factory | List all the triggers that belongs to the Data Factory. |
To use this downloadable Universal Template, you first must perform the following steps:
For the new Universal Task type, create a new task, and enter the task-specific details that were created in the Universal Template.
Field | Description |
Azure Location | Provide the Azure location. |
Tenant ID | Provide the Azure Tenant ID or Directory ID. |
Azure Client ID and Client Secret | Provide the Azure client ID and client secret (Client ID as user name and client secret as the password) - This comes from your Azure App registration services. |
Subscription | Azure Subscription ID. |
Azure Resource Group | Azure Resource group responsible for the Data Factory. |
Data factory Operation | Select the required Azure Data Factory function for the task. |
Data Factory Name | Provide the Azure Data Factory Name. |
Pipeline Name | Pipeline name that needs to be executed in Azure Data factory. |
Pipeline Parameters | Provide the pipeline run time parameters that needs to be passed for the pipeline execution. |
Run ID | Run id for the Azure Data Factory Pipeline execution - Required in case of a restart. |
Restart Pipeline | Check if the pipeline execution needs to be restarted for a failure. |
Start From Failure | Pipeline needs to be restarted from a Failure step. |
Start Activity Name | If Start from Failure is not checked, specify if a failed pipeline needs to be restarted from a specific activity. |
Run ID for Restart | Provide the Pipeline run ID for the restart. |
Trigger Name | Trigger Name in the Azure Data Factory. |
Polling | True for ARMPolling, False for no polling, or a polling object for personal polling strategy. |
This document references the following documents:
Name | Description |
---|---|
User documentation for creating Universal Templates in the Universal Controller user interface. | |
User documentation for creating Universal Tasks in the Universal Controller user interface. |