Skip to content
This repository was archived by the owner on May 14, 2025. It is now read-only.
This repository was archived by the owner on May 14, 2025. It is now read-only.

Streamlined Deployment of ETL Workloads in Spring Cloud Data Flow via Infrastructure as a Code (IaC) #6114

@ionutbalosin

Description

@ionutbalosin

Problem description:
We are starting to experiment with Spring Cloud Data Flow and would like to know the recommended way to deploy workloads (e.g., tasks, batches, and streams) from the CI/CD pipeline. The CLI commands may not be very portable or convenient for us, and frequent changes to the RESTful APIs could require significant effort to upgrade and fix API calls with each new version.

Solution description:
Ideally, we would prefer an SDK to manage deployments. This solution would allow us to define the desired final state of our ETL workloads (composed of tasks, batches, streams, etc.) as code, following the infrastructure as code (IaC) approach, and deploy them via the CI/CD pipeline. This would help us avoid issues related to CLI and RESTful API changes.

Description of alternatives:
We have considered using the CLI commands and RESTful APIs, but these options may not be as portable or convenient, and they could require frequent updates to keep up with changes in new versions.

Additional context:
We are looking for a solution that minimizes the need for manual updates and provides a streamlined process for deploying ETL workloads in a CI/CD pipeline environment.

Metadata

Metadata

Assignees

No one assigned

    Labels

    status/need-triageTeam needs to triage and take a first look

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions