The purpose of this set of prototypes is to provide simple external connection prototypes for both pulsar cli and a java pulsar client. The prototypes first explore unencrypted connections within the Proto-01-kube-basic, and then encrypted connections via tls with cert-manager providing the issuers, certificates and secrets. The final 3rd prototype uses the helm chart for tls deployment.
The original purpose of these learning prototypes was to explore Kubernetes Gateway API with Istio and Cert-Manager for external TLS connections. However, after several failed attempts I determined that Pulsar external connections rely on the Pulsar proxy for providing clients the correct broker. Without a deep dive into the Pulsar Proxy code to figure out how it was handling this, and to then come up with a way for the Gateway API to work within these boundaries I did not see a way forward. As the Proxy is working I will leave the rest for another time when I have more time and a definite need.
| Core Infrastructure | Version |
|---|---|
| Minikube | 1.34.0 |
| Kubernetes | 1.31.0 |
| Docker | 27.2.0 |
| Name | Description |
|---|---|
| Ubuntu | 20.04.6 LTS |
| Processor | Intel® Core™ i7-7700K CPU @ 4.20GHz × 8 |
| Memory | 64 GB |
Instructions for deploying the Core Infrastructure Dependencies listed above are NOT included within this set of Prototypes as there are numerous targeted deployment instructions for each better suited for your particular OS.
| Deployed Name | Version |
|---|---|
| Cert-manager | 1.15.5 |
| Istio | 1.23.2 |
| Kubernetes Gateway API | 1.2.0 |
| Metallb | 0.9.6 |
- Cert-Manager - https://cert-manager.io/docs/
- Istio Ambient Mode - https://istio.io/latest/docs/ambient/
- Kubernetes Gateway API - https://kubernetes.io/docs/concepts/services-networking/gateway/
- Apache Pulsar - https://pulsar.apache.org/docs/4.0.x
The kube based deployments were originally generated by reviewing the output of the helm chart for minikube and tls. The following is the prime example.
helm install --dry-run --values ${PROTODIR}/helm/values-<Proto #>.yaml --namespace pulsar pulsar-mini apache/pulsar > output.txt
It should be noted that minikube has to be running in order to run this dry-run command.
Note :
The commands within the shell files below are meant to be copy pasted (one or a few lines at a time) into a terminal, and not run as an automated bash script.
The Pulsar components deployed within each of the following Prototypes consist of only the required Pulsar components as follows:
- Zookeeper
- Bookkeeper (Bookie)
- Toolset - for Pulsar CLI client
- Broker
- Proxy
Additional components which can be deployed within the Helm chart are excluded from this deployment. These include Prometheus and Graphana as well as helpers such as PodMonitors.
The purpose for this prototype is to provide unencrypted external access to the Pulsar deployment within minikube. The prototype kube yaml components were generated by running a dry-run from the Pulsar minikube Helm chart as follows:
helm install --dry-run --values ${PROTODIR}/helm/values-01.yaml --namespace pulsar pulsar-mini apache/pulsar > kube-pulsar.txt
From this output the kube deployment components were obtained. This script does a minimal installation of only the main required Pulsar components. The pods deployed as listed below.
| Component | # Deployed |
|---|---|
| Zookeeper | 1 |
| Bookie | 1 |
| Toolset | 1 |
| Broker | 1 |
| Proxy | 3 |
The deployment script Step 01 Deploy Minikube, metallb, Pulsar within the scripts folder performs the following:
- Deploys a fresh minikube with minikube addons (dashboard, metallb);
- Configures Metallb loadbalancer
- deploys Pulsar and all its components into the Cluster
- Tests access from the Pulsar CLI client
- Allows running of the simple java test program (eclipse, maven) found in the pulsar-client directory within this project.
The main purpose of this prototype is to provide tls encrypted external access to the Pulsar deployment within minikube. The prototype kube yaml components were generated by running a dry-run from the Pulsar minikube Helm chart as follows:
helm install --dry-run --values ${PROTODIR}/helm/values-02.yaml --namespace pulsar pulsar-mini apache/pulsar > kube-pulsar.txt
From this output the kube deployment components were obtained. This script does a minimal installation of only the main required Pulsar components. The pods deployed as listed below.
| Component | # Deployed |
|---|---|
| Zookeeper | 3 |
| Bookie | 4 |
| Toolset | 1 |
| Broker | 3 |
| Proxy | 3 |
The deployment script Step 01 Deploy Minikube, Metallb,Cert-manager, Pulsar within the scripts folder performs the following:
- Deploys a fresh minikube with minikube addons (dashboard, metallb);
- Configures Metallb loadbalancer
- Deploys Kubernetes Gateway API CRDs (cert-manager deploy uses)
- Deploys istio in Ambient mode.
- Deploys Cert-Manager
- Deploys Pulsar and all required components into the Cluster
- Sets pulsar namespace to istio ambient mode which initiates mTLS between pods
- Tests access from the Pulsar CLI client
- Allows running of the simple java test program (eclipse, maven) found in the pulsar-client directory within this project.
The main purpose of this prototype is to provide tls encrypted external access to the Pulsar deployment within minikube. The deployment uses the Pulsar Helm chart to deploy the following Pulsar components.
| Component | # Deployed |
|---|---|
| Zookeeper | 3 |
| Bookie | 4 |
| Toolset | 1 |
| Broker | 3 |
| Proxy | 3 |
The deployment script Step 01 Deploy Minikube, Metallb,Cert-manager, Pulsar within the scripts folder performs the following:
- Deploys a fresh minikube with minikube addons (dashboard, metallb);
- Configures Metallb loadbalancer
- Deploys Kubernetes Gateway API CRDs (cert-manager deploy uses)
- Deploys istio in Ambient mode.
- Deploys Cert-Manager
- Deploys Pulsar and all required components via the Pulsar Helm chart into the Cluster
- Sets pulsar namespace to istio ambient mode which initiates mTLS between pods
- Tests access from the Pulsar CLI client
- Allows running of the simple java test program (eclipse, maven) found in the pulsar-client directory within this project.