Skip to content

Commit 294e5e0

Browse files
jmagakjmagak
authored
Update Build and Deploy serveless workflows (#1438)
1 parent 43388fd commit 294e5e0

13 files changed

+462
-2
lines changed

‎artifacts/attributes.adoc‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -147,7 +147,8 @@
147147
:monitoring-and-logging-book-title: Monitoring and logging
148148
:observability-category-link: {product-docs-link}/#Observability
149149
:odf-docs-link: link:https://docs.redhat.com/en/documentation/red_hat_openshift_data_foundation/{ocp-version}
150-
:ocp-docs-link: link:https://docs.redhat.com/en/documentation/openshift_container_platform/{ocp-version}
150+
:orchestrator-book-link: {product-docs-link}/html-single/orchestrator_in_red_hat_developer_hub/index
151+
:orchestrator-book-title: Orchestrator in {product}
151152
:osd-docs-link: link:https://docs.redhat.com/en/documentation/openshift_dedicated/{osd-version}
152153
:release-notes-book-link: {product-docs-link}/html-single/red_hat_developer_hub_release_notes/index
153154
:release-notes-book-title: {product} release notes

‎assemblies/assembly-building-and-deploying-serverless-workflows.adoc‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
:_mod-docs-content-type: ASSEMBLY
2+
3+
ifdef::context[]
4+
[id="assembly-building-and-deploying-serverless-workflows"]
5+
endif::[]
6+
:context: orchestrator-rhdh
7+
= Build and deploy serverless workflows
8+
9+
To deploy a workflow and make it available in the Orchestrator plugin, follow these main steps:
10+
11+
* Building workflow images
12+
* Generating workflow manifests
13+
* Deploying workflows to a cluster
14+
15+
This process moves the workflow from your local machine to deployment on a cluster.
16+
17+
// why build workflow images
18+
include::modules/orchestrator/con-benefits-of-workflow-images.adoc[leveloffset=+1]
19+
20+
// project structure
21+
include::modules/orchestrator/con-project-structure-overview.adoc[leveloffset=+2]
22+
23+
// creating and running workflows locally
24+
include::modules/orchestrator/proc-creating-and-running-workflows.adoc[leveloffset=+2]
25+
26+
// building locally and generating artifacts
27+
include::modules/orchestrator/proc-building-locally.adoc[leveloffset=+1]
28+
29+
// the script and its uses
30+
include::modules/orchestrator/con-build-sh-script-and-its-uses.adoc[leveloffset=+2]
31+
32+
// environment variables supported by the script
33+
include::modules/orchestrator/con-environment-variables-supported-by-the-build-script.adoc[leveloffset=+2]
34+
35+
// required tools
36+
include::modules/orchestrator/con-required-tools.adoc[leveloffset=+2]
37+
38+
// building the 01_basic workflow
39+
include::modules/orchestrator/proc-building-the-01-basic-workflow.adoc[leveloffset=+2]
40+
41+
// generated workflow manifests
42+
include::modules/orchestrator/con-generated-workflow-manifests.adoc[leveloffset=+1]
43+
44+
// deploy workflows on a cluster
45+
include::modules/orchestrator/proc-deploying-workflows-on-a-cluster.adoc[leveloffset=+1]

‎modules/orchestrator/con-benefits-of-workflow-images.adoc‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
:_mod-docs-content-type: CONCEPT
2+
3+
[id="con-benefits-of-workflow-images.adoc_{context}"]
4+
= Benefits of workflow images
5+
6+
While the OpenShift Serverless Logic Operator supports the building of workflows dynamically, this approach is primarily for experimentation.
7+
For production deployments, building images is the preferred method due to the following reasons:
8+
9+
* Production readiness: Prebuilt images can be scanned, secured, and tested before going live.
10+
* GitOps compatibility: The Orchestrator relies on a central OpenShift Serverless Logic Operator instance to track workflows and their state. To use this tracking service, you must deploy workflows with the `gitops` profile, which expects a prebuilt image.
11+
* Testing and quality: Building an image gives you more control over the testing process.

‎modules/orchestrator/con-build-sh-script-and-its-uses.adoc‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
:_mod-docs-content-type: CONCEPT
2+
3+
[id="con-build-sh-script-and-its-uses.adoc_{context}"]
4+
= The `build-sh` script functionality and important flags
5+
6+
The `build-sh` script does the following tasks in order:
7+
8+
* Generates workflow manifests using the `kn-workflow` CLI.
9+
* Builds the workflow image using `podman` or `docker`.
10+
* Optional: The script pushes the images to an image registry and deploys the workflow using `kubectl`.
11+
12+
You can review the script configuration options and see available flags and their functions by accessing the help menu:
13+
14+
[source,bash]
15+
----
16+
./scripts/build.sh [flags]
17+
----
18+
19+
The following flags are essential for running the script:
20+
21+
[cols="1,3", options="header"]
22+
|===
23+
|Flag |Description
24+
|`-i`, `--image` |Required: Full image path, for example, `quay.io/orchestrator/demo:latest`
25+
|`-w`, `--workflow-directory` |Workflow source directory (default is the current directory)
26+
|`-m`, `--manifests-directory` |Where to save generated manifests
27+
|`--push` |Push the image to the registry
28+
|`--deploy` |Deploy the workflow
29+
|`-h`, `--help` |Show the help message
30+
|===
31+
32+
[TIP]
33+
====
34+
The script also supports builder and runtime image overrides, namespace targeting, and persistence flags.
35+
====

‎modules/orchestrator/con-environment-variables-supported-by-the-build-script.adoc‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
:_mod-docs-content-type: CONCEPT
2+
3+
[id="con-environment-variables-supported-by-the-build-script.adoc_{context}"]
4+
= Environment variables supported by the build script
5+
6+
The `build-sh` script supports the following environment variables to customize the workflow build process without modifying the script itself:
7+
8+
`QUARKUS_EXTENSIONS`::
9+
10+
The `QUARKUS_EXTENSIONS` variable specifies additional link:https://quarkus.io/extensions/[Quarkus] extensions required by the workflow. This variable takes the format of a comma-separated list of fully qualified extension IDs as shown in the following example:
11+
+
12+
[source,yaml]
13+
----
14+
export QUARKUS_EXTENSIONS="io.quarkus:quarkus-smallrye-reactive-messaging-kafka"
15+
----
16+
+
17+
Add Kafka messaging support or other integrations at build time.
18+
19+
`MAVEN_ARGS_APPEND`::
20+
21+
The `MAVEN_ARGS_APPEND` variable appends additional arguments to the *Maven build* command. This variable takes the format of a string of Maven CLI arguments as shown in the following example:
22+
+
23+
[source,yaml]
24+
----
25+
export MAVEN_ARGS_APPEND="-DmaxYamlCodePoints=35000000"
26+
----
27+
+
28+
Control build behavior. For example, set `maxYamlCodePoints` parameter that controls the maximum input size for YAML input files to 35000000 characters (~33MB in UTF-8).

‎modules/orchestrator/con-generated-workflow-manifests.adoc‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
:_mod-docs-content-type: CONCEPT
2+
3+
[id="con-generated-workflow-manifests.adoc_{context}"]
4+
= Generated workflow manifests
5+
6+
The following example is an illustration of what is generated under the `01_basic/manifests`:
7+
8+
[source,yaml]
9+
----
10+
01_basic/manifests
11+
├── 00-secret_basic-secrets.yaml
12+
├── 01-configmap_basic-props.yaml
13+
├── 02-configmap_01-basic-resources-schemas.yaml
14+
└── 03-sonataflow_basic.yaml
15+
----
16+
17+
`00-secret_basic-secrets.yaml`::
18+
Contains secrets from `01_basic/src/main/resources/secret.properties`.
19+
Values are not required at this stage as you can set them later after applying CRs or when using GitOps.
20+
21+
[Important]
22+
====
23+
In OpenShift Serverless Logic `v1.36`, after updating a secret, you must manually restart the workflow Pod for changes to apply.
24+
====
25+
26+
`01-configmap_basic-props.yaml`::
27+
Holds application properties from application.properties.
28+
Any change to this ConfigMap triggers an automatic Pod restart.
29+
30+
`02-configmap_01-basic-resources-schemas.yaml`::
31+
Contains JSON schemas from src/main/resources/schemas.
32+
+
33+
[NOTE]
34+
====
35+
You do not need to deploy certain configuration resources when using the GitOps profile.
36+
====
37+
38+
`03-sonataflow_basic.yaml`::
39+
The SonataFlow custom resource (CR) that defines the workflow.
40+
+
41+
[source,yaml]
42+
----
43+
podTemplate:
44+
container:
45+
image: quay.io/orchestrator/demo-basic
46+
resources: {}
47+
envFrom:
48+
- secretRef:
49+
name: basic-secrets
50+
----
51+
+
52+
[source,yaml,subs="+quotes"]
53+
----
54+
persistence:
55+
postgresql:
56+
secretRef:
57+
name: `sonataflow-psql-postgresql`
58+
userKey: `__<your_postgres_username>__`
59+
passwordKey: `__<your_postgres_password>__`
60+
serviceRef:
61+
name: `sonataflow-psql-postgresql`
62+
port: 5432
63+
databaseName: sonataflow
64+
databaseSchema: basic
65+
----
66+
+
67+
where:
68+
69+
`postgresql:secretRef:name`:: Enter the Secret name for your deployment.
70+
`postgresql:secretRef:userKey`:: Enter the key for your deployment.
71+
`postgresql:secretRef:passwordKey`:: Enter the password for your deployment.
72+
`postgresql:serviceRef:name`:: Enter the Service name for your deployment.
73+
+
74+
If you must connect to an external database, replace `serviceRef` with `jdbcUrl`. See link:https://docs.redhat.com/en/documentation/red_hat_openshift_serverless/1.36/html-single/serverless_logic/index#serverless-logic-managing-persistence[Managing workflow persistence].
75+
76+
By default, the script generates all the manifests without a namespace. You can specify a namespace to the script by using the `--namespace` flag if you know the target namespace in advance. Otherwise, you must provide the namespace when applying the manifests to the cluster. See link:https://docs.redhat.com/en/documentation/red_hat_openshift_serverless/1.36/html-single/serverless_logic/index#serverless-logic-configuring-workflow-services[Configuring workflow services].

‎modules/orchestrator/con-project-structure-overview.adoc‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
:_mod-docs-content-type: CONCEPT
2+
3+
[id="con-project-structure-overview.adoc_{context}"]
4+
= Project structure overview
5+
6+
The project utilizes *Quarkus project layout* (Maven project structure). This structure is illustrated by the following `01_basic` workflow example:
7+
8+
[source, yaml]
9+
----
10+
01_basic
11+
├── pom.xml
12+
├── README.md
13+
└── src
14+
└── main
15+
├── docker
16+
│ ├── Dockerfile.jvm
17+
│ ├── Dockerfile.legacy-jar
18+
│ ├── Dockerfile.native
19+
│ └── Dockerfile.native-micro
20+
└── resources
21+
├── application.properties
22+
├── basic.svg
23+
├── basic.sw.yaml
24+
├── schemas
25+
│ ├── basic__main-schema.json
26+
│ └── workflow-output-schema.json
27+
└── secret.properties
28+
----
29+
30+
The main workflow resources are located under the `src/main/resources/` directory.
31+
32+
The `kn-workflow CLI` generated this project structure. You can try generating the structure yourself by following the _Getting Started guide_. For more information on the Quarkus project, see link:https://quarkus.io/guides/getting-started[Creating your first application].

‎modules/orchestrator/con-required-tools.adoc‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
:_mod-docs-content-type: CONCEPT
2+
3+
[id="con-required-tools.adoc_{context}"]
4+
= Required tools
5+
6+
To run the `build-sh` script locally and manage the workflow lifecycle, you must install the following command-line tools:
7+
8+
[cols="1,3", options="header"]
9+
|===
10+
|Tool |Conceptual Purpose.
11+
|podman or docker |Container runtime required for building the workflow images.
12+
|`kubectl` |Kubernetes CLI.
13+
|`yq` |YAML processor.
14+
|`jq` |JSON processor.
15+
|`curl`, `git`, `find`, `which`| Shell utilities.
16+
|`kn-workflow` |CLI for generating workflow manifests.
17+
|===

‎modules/orchestrator/proc-building-locally.adoc‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
:_mod-docs-content-type: PROCEDURE
2+
3+
[id="proc-building-locally.adoc_{context}"]
4+
= Building workflow images locally
5+
6+
You can use the build script (`build.sh`) to build workflow images. You can run it either locally or inside a container. This section highlights how build workflow images locally.
7+
8+
.Procedure
9+
10+
. Clone the project as shown in the following example:
11+
+
12+
[source, yaml]
13+
----
14+
git clone git@github.com:rhdhorchestrator/orchestrator-demo.git
15+
cd orchestrator-demo
16+
----
17+
18+
. Check the help menu of the script:
19+
+
20+
[source,yaml]
21+
----
22+
./scripts/build.sh --help
23+
----
24+
25+
. Run the `build.sh` script, providing the required flags, for instance, the image path (`-i`), workflow source directory (`-w`), and manifests output directory (`-m`).
26+
27+
+
28+
[IMPORTANT]
29+
====
30+
You must specify the full target image path with a tag as shown in the following example:
31+
[source,yaml]
32+
----
33+
./scripts/build.sh --image=quay.io/orchestrator/demo-basic:test -w 01_basic/ -m 01_basic/manifests
34+
----
35+
====

‎modules/orchestrator/proc-building-the-01-basic-workflow.adoc‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
:_mod-docs-content-type: PROCEDURE
2+
3+
[id="proc-building-the-01-basic-workflow.adoc_{context}"]
4+
= Building the `01_basic` workflow
5+
6+
To run the script from the root directory of the repository, you must use the `-w` flag to point to the workflow directory. Additionally, specify the output directory with the `-m` flag.
7+
8+
.Prerequisites
9+
10+
* You have specified the target image using a tag.
11+
12+
.Procedure
13+
14+
. Run the following command:
15+
+
16+
[source,bash]
17+
----
18+
./scripts/build.sh --image=quay.io/orchestrator/demo-basic:test -w 01_basic/ -m 01_basic/manifests
19+
----
20+
+
21+
This build command produces the following two artifacts:
22+
23+
* A workflow image and Kubernetes manifests: `quay.io/orchestrator/demo-basic:test` and tagged as `latest`.
24+
* Kubernetes manifests under: `01_basic/manifests/`
25+
26+
. Optional: You can add the `--push` flag to automatically push the image after building. Otherwise, pushing manually is mandatory before deploying.
27+

0 commit comments

Comments
 (0)