Skip to main content

Airbyte

Disclaimer​

Your use of this download is governed by Stonebranch's Terms of Use.

Version Information​

Template Name

Extension Name

Version

Status

Airbyte

ue-airbyte

1 (Current 1.0.0)

Initial version.

Refer to the Changelog for version history information.

Overview​

Airbyte is an open-source data integration platform designed to enable organizations to collect, centralize, and sync data across various systems and databases. It provides connectors for integrating a wide variety of data sources (such as databases, APIs, and file systems) into destinations like data warehouses, lakes, and other cloud storage solutions.

Universal Automation Center (UAC) can trigger Airbyte jobs by making API requests to the Airbyte API endpoint with appropriate authentication, enabling automated execution of your data transformation workflows based on your orchestration requirements.

Key Features​

Feature

Description

Job Triggering

Trigger and monitor Airbyte Jobs.

Tunable Output Options

STDOUT: Capability to show "Job Info"

Requirements​

This integration requires a Universal Agent and a Python runtime to execute the Universal Task.

Area

Details

Python Version

Requires Python 3.11, Tested with Agent bundled python distribution.

Universal Agent Compatibility

  • Compatible with Universal Agent for Windows x64 and version >= 7.6.0.0.
  • Compatible with Universal Agent for Linux and version >= 7.6.0.0.

Universal Controller Compatibility

Universal Controller Version >= 7.6.0.0.

Network and Connectivity

Connectivity to Airbyte APIs is required.

Airbyte

Compatible with Airbyte API v1.

Supported Actions​

Action: Trigger Job​

The "Trigger Job" action provides the following capabilities.

  • Trigger or Trigger and Monitor an Airbyte Job waiting to reach a final state, providing the required input parameters for execution.
  • Control the information to be displayed on STDOUT, to bring visibility on the executed job info.

Action Output​

The extension output provides the following information:

  • exit_code, status_description: General info regarding the task execution. For more information users can refer to the exit code table.
  • invocation.fields: The task configuration used for this task execution.
  • result.job_info: Shows basic information about the Job.
    • result.job_id
    • result.status
    • result.job_type
    • result.connection_id
    • result.start_time
    • result.last_updated_at
      • Only shown when Wait for Success or Failure is True.
    • result.duration
      • Only shown when Wait for Success or Failure is True.
  • result.errors: List of errors that might have occurred during execution. That does not include Airbyte Job internal errors.

Examples:

Success: Job Triggering without Monitoring
{
    "exit_code": 0,
    "status_description": "Task executed successfully",
    "invocation": {
        "extension": "ue-airbyte",
        "version": "1.0.0",
        "fields": {
            "action": "Trigger Job",
            "job_type": "Sync",
            "airbyte_api_url": "http://api.airbyte.com",
            "airbyte_credentials": {
                "user": "107b5b4d-1234-4567-bb20-abc1239b5123",
                "password": "****",
                "token": "****",
                "key_location": null,
                "passphrase": "****"
            },
            "workspace_name": "Default Workspace",
            "connection_name": "Postgres → MySQL (ID: c798cc11-20f1-4e60-9a0d-30ae22d2f87e)",
            "wait": true,
            "poll_interval": 60,
            "max_polls": 0,
            "source_id": null,
            "destination_id": null,
            "stdout_options": "Include Job Info",
            "job_id": 87,
            "job_status": null,
            "job_start_time": null,
            "last_updated_on": null,
            "job_duration": null
        }
    },
    "result": {
        "job_info": {
            "job_id": 87,
            "status": "succeeded",
            "job_type": "sync",
            "connection_id": "c798cc11-20f1-4e60-9a0d-30ae22d2f87e",
            "start_time": "March 24, 2026 at 13:25:42",
            "last_updated_at": "March 24, 2026 at 14:02:01",
            "duration": "0:36:19"
        }
    }
}
Fail: Job Already Running
{
    "exit_code": 1,
    "status_description": "AirbyteAPIError: trigger_sync failed: A sync is already running for: c798cc11-20f1-4e60-9a0d-30ae22d2f87e",
    "invocation": {
        "extension": "ue-airbyte",
        "version": "1.0.0",
        "fields": {
            ...
        }
    },
    "result": {
        "errors": [
            "AirbyteAPIError: trigger_sync failed: A sync is already running for: c798cc11-20f1-4e60-9a0d-30ae22d2f87e"
        ]
    }
}

Configuration Examples​

Example: Trigger an Airbyte​

The Airbyte task triggers an Airbyte job with the name "Postgres → MySQL" with no monitoring. The task will end when it triggers the job run on Airbyte, and the task will not wait for it to finish. If the Job is triggered successfully, the task instance completes with success, else with failure.

image-20260324-123220.png

Example: Trigger and Monitor an Airbyte Job​

The Airbyte task will trigger an Airbyte with the name "Postgres → MySQL" with monitoring. The "Include Job Info" STDOUT options are selected. The task will remain running until the job run on Airbyte reaches a terminal state.

The task will check if the job run has reached a terminal state on each poll interval. Once the task reaches a terminal state, it will print the Job Info to STDOUT. Then the task will end with success or failure, depending on the terminal state of the Job Run.

image-20260324-123834.png

Input Fields​

Name

Type

Description

Version Information

Action

Choice

The action performed upon the task execution.

  • Trigger Job (default)

Introduced in 1.0.0

Airbyte API URL

Text

The base API URL of your service (e.g., https://api.airbyte.com )

Introduced in 1.0.0

Airbyte Credentials

Credentials

Credentials for the interaction with Airbyte API.

The Credential definition should be as follows.

  • Airbyte Application Client ID as "Runtime User".
  • Airbyte Application Client Secret as "Runtime Password".

Introduced in 1.0.0

Workspace

Dynamic Choice

Dynamic Choice field for the retrieval and selection of an Airbyte Workspace.

Used for filtering Airbyte Connections which belong to the selected Workspace when executing the Connection dynamic choice field.

Introduced in 1.0.0

Connection

Dynamic Choice

Dynamic Choice field for the retrieval and selection of the Airbyte Connection on which to trigger a job.

Only shows Connections which belong to the selected Workspace.

Introduced in 1.0.0

Job Type

Choice

The type of job to trigger.

Available options are as follows.

  • Sync
  • Reset

Introduced in 1.0.0

Wait For Success Or Failure

Checkbox

When checked, the Task Instance waits until the triggered Job reaches a final State. Otherwise, the Task Instance completes when the Job is simply triggered, without waiting to finish.

Introduced in 1.0.0

Poll Interval (sec)

Integer

This field is required when Wait For Success Or Failure is checked and represents the time interval between retries for getting the status of the Airbyte Job.

As a best practice, if the Task Execution expected completion duration is long, set the polling Interval to a larger value. A short value will trigger frequent checks towards Airbyte, which is inefficient in terms of resources in the case of long-duration Tasks.

The default value is 60.

Introduced in 1.0.0

Max Polls

Integer

This field is required when Wait For Success Or Failure is checked and represents the maximum number of polls to do while waiting for a terminal state before failing.

If set to 0, the task will poll indefinitely.

Introduced in 1.0.0

STDOUT Options

Choice

Controls what information is displayed on the STDOUT. Allowed values:

  • Include Job Info

Introduced in 1.0.0

Output Fields​

Name

Type

Description

Version Information

Job ID

Integer

The Job ID of the Triggered Job.

Introduced in 1.0.0.

Job Status

Text

The last retrieved Job Status.

Introduced in 1.0.0.

Job Start Time

Text

The timestamp of when the Job started on Airbyte.

Introduced in 1.0.0.

Job Last Updated On

Text

The timestamp of the most recent update made to the Job.

Gets value only if Wait For Success Or Failure is checked.

Introduced in 1.0.0.

Job Duration

Text

The total amount of time the Job took to execute.

Gets value only if Wait For Success Or Failure is checked.

Introduced in 1.0.0.

Environment Variables​

Environment Variables can be set from the Environment Variables task definition table. The following environment variables can affect the behavior of the extension.

Environment Variable

Description

Version Information

UE_HTTP_TIMEOUT

Specifies the timeout (in seconds) for HTTP requests made by the Task Instance. A higher value allows for slower responses, while a lower value enforces stricter time constraints. If not set, a default of 60 seconds is used. When a timeout happens, the Task Instance ends in failure.

Introduced in 1.0.0.

Cancellation and Rerun​

  • In the case of cancellation of the Task Instance, the Job remains intact on Airbyte.
  • In the case of a re-run of the Task Instance, a new Job is created.
  • In the case of the dynamic command, Cancel Job is being executed. The Job will be cancelled on Airbyte.

Dynamic Commands​

Cancel Job​

Dynamic Command "Cancel Job" is available in the "Running" and "Success" states of the task instance, and is used to cancel the Job on Airbyte.

Exit Codes​

Exit Code

Status

Status Description

Meaning

0

Success

If the Wait For Success Or Failure input field is checked,

  • "Task executed successfully, job monitoring finished. The job run on dbt Cloud finished successfully"

If the Wait For Success Or Failure input field is not checked,

  • "Task executed successfully, job '<<Job Name>>' run on dbt Cloud triggered successfully"

Successful Execution.

If Wait For Success Or Failure input field is checked,

  • It means that the Job Run ended up in a Success State.

If Wait for Success or Failure input field is not checked,

  • It means that the Flow Run has been successfully triggered.

1

Failure

"Execution Failed: <<Error Description>>"

Failed Execution.

  • If Wait For Success Or Failure input field is checked, it means that the Flow Run ended up in a Failed State.
  • This error code also acts as a generic error code. Raised when an error does not fall into the other error scenarios.

2

Cancelled

"Task instance cancelled successfully"

Cancelled Execution.

  • If the task is cancelled via the Cancel button in the right click menu of the task instance.

20

Failure

"Data Validation Error: <<Error Description>>"

Input fields validation error.

40

Failure

"MaxPollsReached: <<Error Description>>"

Maximum polls reached.

100

Success

"Job Cancelled"

Job has been cancelled successfully.

STDOUT and STDERR​

STDOUT is used for displaying Job information, and it's controlled by STDOUT Options.

STDERR provides additional information to the user, the verbosity of which is controlled by the Log Level Task Definition field.

info

Backwards compatibility is not guaranteed for the content of STDOUT/STDERR and can be changed in future versions without notice

How To​

Import Universal Template​

To use the Universal Template, you first must perform the following steps.

  1. This Universal Task requires the Resolvable Credentials feature. Check that the Resolvable Credentials Permitted system property has been set to true.
  2. Import the Universal Template into your Controller:
    1. Extract the zip file that you downloaded from the Integration Hub.
    2. In the Controller UI, select Services > Import Integration Template option.
    3. Browse to the "export" folder under the extracted files for the ZIP file (Name of the file will be unv_tmplt_*.zip) and click Import.
    4. When the file is imported successfully, refresh the Universal Templates list; the Universal Template will appear on the list.

Modifications of this integration, applied by users or customers, before or after import, might affect the supportability of this integration. For more information, refer to Integration Modifications.

Configure Universal Task​

For a new Universal Task, create a new task and enter the required input fields.

Integration Modifications​

Modifications applied by users or customers, before or after import, might affect the supportability of this integration. The following modifications are discouraged to retain the support level as applied for this integration.

  • Python code modifications should not be done.
  • Template Modifications
    • General Section
      • "Name", "Extension", "Variable Prefix", and "Icon" should not be changed.
    • Universal Template Details Section
      • "Template Type", "Agent Type", "Send Extension Variables", and "Always Cancel on Force Finish" should not be changed.
    • Result Processing Defaults Section
      • Success and Failure Exit codes should not be changed.
      • Success and Failure Output processing should not be changed.
    • Fields Restriction Section
      The setup of the template does not impose any restrictions. However, concerning the "Exit Code Processing Fields" section.
      1. Success/Failure exit codes need to be respected.
      2. In principle, as STDERR and STDOUT outputs can change in follow-up releases of this integration, they should not be considered as a reliable source for determining the success or failure of a task.

Users and customers are encouraged to report defects or feature requests at the Stonebranch Support Desk.

Document References​

This document references the following documents:

Document Link

Description

Universal Templates

User documentation for creating, working with and understanding Universal Templates and Integrations.

Universal Tasks

User documentation for creating Universal Tasks in the Universal Controller user interface.

Airbyte API Documentation

The Airbyte API Documentation

Changelog​

ue-airbyte-1.0.0 (2025-08-01)​

Initial Version