Your use of this download is governed by Stonebranch’s Terms of Use, which are available at https://www.stonebranch.com/integration-hub/Terms-and-Privacy/Terms-of-Use/
Pentaho Data Integration provides powerful ETL (Extract, Transform and Load) capabilities. Universal Controller is integrated to orchestrate the jobs and transformations within Pentaho Data Integration platform via Carte webservice calls.
As an alternative approach, if there is no Carte webserver set-up, it is possible to use the Kitchen & Pan command line utility to execute the jobs & Transformations within the Pentaho Data Integration platform via Windows or Linux Task type in Universal Controller.
The current integration with Pentaho is developed using the Universal Extension functionality in Universal Controller.
The following is the overview of this integration.
UAC communicates with Pentaho Data Integration through the REST-API methods available for the Carte Server.
Universal Controller offers the following functionalities that can be orchestrated within the Pentaho Data Integration Platform:
Run a Pentaho Data Integration Job where the repository details are already supplied in Carte configuration files.
Run a Pentaho Data Integration Job from any of the available repository Via Carte web service calls.
Run a Pentaho Data Integration Job from a *.KJB file.
Define & Run a Pentaho Integration Job from Universal Controller Script library.
Execute a Pentaho Data Integration Transformation where the repository details are already supplied in Carte configuration files.
Run a Pentaho Data Integration Transformation from a *.KTB file.
Define & Run a Pentaho Integration Transformation from Universal Controller Script library.
This integration requires a Universal Agent 7.0 and a Python runtime to execute the Universal Task against a Pentaho Data Integration.
Universal Agent from V7.0 supporting Universal Extension
Requires Python 3.6 or higher. Tested with the Universal Agent bundled Python distribution.
Python modules required:
requests
base64
ElementTree
Universal Agent for Linux or Windows Version 7.0.0.0 or later is required.
Universal Agent needs to be installed with python option (--python yes).
Universal Controller Version 7.0.0.0 and later.
This Universal Task has been tested with the following Pentaho Data Integration versions:
9.1
Versions from 8.0 can also be supported as it have the same Carte REST-API
Universal Agent and Universal Controller from Version 7.0 and later.
Installation of a Universal Agent V7.0 in Pentaho Server.
Pentaho tool must be configured with the Carte Server configuration.
Necessary Python modules installed in the Universal Agent Python library as described in the section above.
Feature | Description |
Run a Job from Carte configured repository | Trigger a Pentaho Data Integration Job execution ,Where the repository details are already supplied in Carte configuration files. |
Run a Job from a Repository | Trigger a Pentaho Data Integration Job execution , Where the Pentaho Repository details can be supplied as a Input in Universal Controller Job definition. |
Run Job from file | Run a Pentaho Data Integration Job from a *.KJB file located in the remote host. |
Run a Job from UAC Script Library | Define & Run a Pentaho Integration Job from Universal Controller Script library. |
Execute a Transformation from a Repository | Trigger a Pentaho Data Integration Transformation execution ,Where the repository details are already supplied in Carte configuration files. |
Execute a Transformation from a file | Run a Pentaho Data Integration Job from a *.KTB file located in the remote host. |
Run a Transformation from UAC Library | Define & Run a Pentaho Integration Job from Universal Controller Script library. |
To use the Pentaho Universal Extension, you first must perform the following steps:
When the template has been imported successfully, the Universal Template will appear on the list. Refresh your Navigation Tree to see these tasks in the Automation Center Menu.
For the new Universal Task type, create a new task, and enter the task-specific details that were created in the Universal Template.
Field | Description |
Select a Pentaho Function | Select the desired function you would need to perform in Pentaho |
Carte server URL | Provide the Carte webserver URL configured in the Pentaho Platform |
Carte Credentials | Provide the Carte webserver Credentials for Jobs/Transformation webservice calls |
SSL Verify(Carte API Call) | Check if carte web server URL needs to be SSL Verified |
Path to SSL Certificate | Provide the path and file name of the SSL Certificate (Needs to be in the utility agent file system) |
Is Agent Installed in Carte Server | If you need run a transformation or Job from UAC library then Universal agent needs to be installed in the Pentaho Carte server |
Select Pentaho Job or Transformation Script | Select a XML script for Pentaho Job or Transformation from UAC script library |
Repository Name | Provide the Pentaho Repository for the Job to be executed |
Pentaho Repository Credentials | Provide the Pentaho Repository credentials for the Job Execution REST-API call |
Pentaho Job Name | Provide the exact Pentaho job name with path incase of repository and just the job name incase of a file |
Job File(Path & Name) | Provide the Pentaho job file name with path and this file should reside in the carte server |
Transformation Name | Provide the Pentaho transformation name (*Ensure a unique transformation name when Execute a Transformation from a file) |
Transformation File Path & Name | Provide the Pentaho Transformation file name and path |
Input Parameters(if amy) | Add if any input parameters to be used for job execution starting with '&' and every parameters to be separated with ampersand symbol(&) |
Log Level | Select either INFO or DEBUG OR WARN OR ERROR |
Execution ID | Output only field : Displays the Carte execution ID of transformation triggered from Universal Controller |
Status | Output only field : Status of the Job/Transformation executing in Carte Web Server |
Print Job Log | Check Box if the Job/Transformation execution logs to be printed in Universal Controller STDERR |
This document references the following documents:
Name | Location | Description |
---|---|---|
Universal Templates | User documentation for creating Universal Templates in the Universal Controller user interface. | |
Universal Tasks | User documentation for creating Universal Tasks in the Universal Controller user interface. | |
Universal Extension | User documentation for creating Universal Extension Tasks in the Universal Controller user interface. | |
Pentaho Rest-API Reference | https://help.hitachivantara.com/Documentation/Pentaho/9.1/Developer_center/REST_API_Reference | Pentaho Developer REST-API Reference |
Your use of this download is governed by Stonebranch’s Terms of Use, which are available at https://www.stonebranch.com/integration-hub/Terms-and-Privacy/Terms-of-Use/
Pentaho Data Integration provides powerful ETL (Extract, Transform and Load) capabilities. Universal Controller is integrated to orchestrate the jobs and transformations within Pentaho Data Integration platform via Carte webservice calls.
As an alternative approach, if there is no Carte webserver set-up, it is possible to use the Kitchen & Pan command line utility to execute the jobs & Transformations within the Pentaho Data Integration platform via Windows or Linux Task type in Universal Controller.
The current integration with Pentaho is developed using the Universal Extension functionality in Universal Controller.
The following is the overview of this integration.
UAC communicates with Pentaho Data Integration through the REST-API methods available for the Carte Server.
Universal Controller offers the following functionalities that can be orchestrated within the Pentaho Data Integration Platform:
Run a Pentaho Data Integration Job where the repository details are already supplied in Carte configuration files.
Run a Pentaho Data Integration Job from any of the available repository Via Carte web service calls.
Run a Pentaho Data Integration Job from a *.KJB file.
Define & Run a Pentaho Integration Job from Universal Controller Script library.
Execute a Pentaho Data Integration Transformation where the repository details are already supplied in Carte configuration files.
Run a Pentaho Data Integration Transformation from a *.KTB file.
Define & Run a Pentaho Integration Transformation from Universal Controller Script library.
This integration requires a Universal Agent 7.0 and a Python runtime to execute the Universal Task against a Pentaho Data Integration.
Universal Agent from V7.0 supporting Universal Extension
Requires Python 3.6 or higher. Tested with the Universal Agent bundled Python distribution.
Python modules required:
requests
base64
ElementTree
Universal Agent for Linux or Windows Version 7.0.0.0 or later is required.
Universal Agent needs to be installed with python option (--python yes).
Universal Controller Version 7.0.0.0 and later.
This Universal Task has been tested with the following Pentaho Data Integration versions:
9.1
Versions from 8.0 can also be supported as it have the same Carte REST-API
Universal Agent and Universal Controller from Version 7.0 and later.
Installation of a Universal Agent V7.0 in Pentaho Server.
Pentaho tool must be configured with the Carte Server configuration.
Necessary Python modules installed in the Universal Agent Python library as described in the section above.
Feature | Description |
Run a Job from Carte configured repository | Trigger a Pentaho Data Integration Job execution ,Where the repository details are already supplied in Carte configuration files. |
Run a Job from a Repository | Trigger a Pentaho Data Integration Job execution , Where the Pentaho Repository details can be supplied as a Input in Universal Controller Job definition. |
Run Job from file | Run a Pentaho Data Integration Job from a *.KJB file located in the remote host. |
Run a Job from UAC Script Library | Define & Run a Pentaho Integration Job from Universal Controller Script library. |
Execute a Transformation from a Repository | Trigger a Pentaho Data Integration Transformation execution ,Where the repository details are already supplied in Carte configuration files. |
Execute a Transformation from a file | Run a Pentaho Data Integration Job from a *.KTB file located in the remote host. |
Run a Transformation from UAC Library | Define & Run a Pentaho Integration Job from Universal Controller Script library. |
To use the Pentaho Universal Extension, you first must perform the following steps:
When the template has been imported successfully, the Universal Template will appear on the list. Refresh your Navigation Tree to see these tasks in the Automation Center Menu.
For the new Universal Task type, create a new task, and enter the task-specific details that were created in the Universal Template.
Field | Description |
Select a Pentaho Function | Select the desired function you would need to perform in Pentaho |
Carte server URL | Provide the Carte webserver URL configured in the Pentaho Platform |
Carte Credentials | Provide the Carte webserver Credentials for Jobs/Transformation webservice calls |
SSL Verify(Carte API Call) | Check if carte web server URL needs to be SSL Verified |
Path to SSL Certificate | Provide the path and file name of the SSL Certificate (Needs to be in the utility agent file system) |
Is Agent Installed in Carte Server | If you need run a transformation or Job from UAC library then Universal agent needs to be installed in the Pentaho Carte server |
Select Pentaho Job or Transformation Script | Select a XML script for Pentaho Job or Transformation from UAC script library |
Repository Name | Provide the Pentaho Repository for the Job to be executed |
Pentaho Repository Credentials | Provide the Pentaho Repository credentials for the Job Execution REST-API call |
Pentaho Job Name | Provide the exact Pentaho job name with path incase of repository and just the job name incase of a file |
Job File(Path & Name) | Provide the Pentaho job file name with path and this file should reside in the carte server |
Transformation Name | Provide the Pentaho transformation name (*Ensure a unique transformation name when Execute a Transformation from a file) |
Transformation File Path & Name | Provide the Pentaho Transformation file name and path |
Input Parameters(if amy) | Add if any input parameters to be used for job execution starting with '&' and every parameters to be separated with ampersand symbol(&) |
Log Level | Select either INFO or DEBUG OR WARN OR ERROR |
Execution ID | Output only field : Displays the Carte execution ID of transformation triggered from Universal Controller |
Status | Output only field : Status of the Job/Transformation executing in Carte Web Server |
Print Job Log | Check Box if the Job/Transformation execution logs to be printed in Universal Controller STDERR |
This document references the following documents:
Name | Location | Description |
---|---|---|
Universal Templates | User documentation for creating Universal Templates in the Universal Controller user interface. | |
Universal Tasks | User documentation for creating Universal Tasks in the Universal Controller user interface. | |
Universal Extension | User documentation for creating Universal Extension Tasks in the Universal Controller user interface. | |
Pentaho Rest-API Reference | https://help.hitachivantara.com/Documentation/Pentaho/9.1/Developer_center/REST_API_Reference | Pentaho Developer REST-API Reference |