Much better Pro docs #4263
Much better Pro docs #4263
Conversation
github-actions
bot
added
internal
Documentation Link Check Results❌ Absolute links check failed |
🔍 Broken Links ReportSummary
Details
📂 Full file paths
|
| 1. **Code Execution**: You write code and run pipelines with your client SDK using Python | ||
| 2. **Authentication & Token Acquisition**: | ||
| - Users authenticate via your internal identity provider (LDAP/AD/OIDC) | ||
| - The ZenML Pro control plane (running in your infrastructure) handles authentication and RBAC | ||
| - The ZenML client fetches short-lived tokens from your ZenML workspace for: | ||
| - Pushing Docker images to your container registry | ||
| - Communicating with your artifact store | ||
| - Submitting workloads to your orchestrator | ||
| - *Note: Your local Python environment needs the client libraries for your stack components* | ||
| 3. **Authorization**: RBAC policies enforced by your control plane before token issuance | ||
| 4. **Image & Workload Submission**: The client pushes Docker images (and optionally code if no code repository is configured) to your container registry, then submits the workload to your orchestrator | ||
| 5. **Orchestrator Execution**: In the orchestrator environment within your infrastructure: | ||
| - The Docker image is pulled from your container registry | ||
| - Within the pipeline/step entrypoint, the necessary code is pulled in | ||
| - A connection to your ZenML workspace is established | ||
| - The relevant pipeline/step code is executed | ||
| 6. **Runtime Data Flow**: During execution (all within your infrastructure): | ||
| - Pipeline and step run metadata is logged to your ZenML workspace | ||
| - Logs are streamed to your log backend | ||
| - Artifacts are written to your artifact store | ||
| - Metadata pointing to these artifacts is persisted in your workspace | ||
| 7. **Observability**: The ZenML Pro dashboard (running in your infrastructure) connects to your workspace and uses all persisted metadata to provide you with a complete observability plane |
|
|
||
| The diagram above illustrates a complete air-gapped ZenML Pro deployment with all components running within your organization's VPC. This architecture ensures zero external communication while providing full enterprise MLOps capabilities. | ||
|
|
||
| ### Architecture Components |
| - **Backup sites** for disaster recovery | ||
| - **Monitoring and alerting** for all components | ||
|
|
||
| ## Pre-requisites |
|
https://zenml-io.gitbook.io/alexej/zenml-pro - view here to see it in action |
htahir1
left a comment
htahir1
left a comment
There was a problem hiding this comment.
I think its good for a first round. many comments apply to many pages
| - ✅ **Vulnerability Assessment Reports** available on request | ||
| - ✅ **Software Bill of Materials (SBOM)** available on request |
There was a problem hiding this comment.
yes, we can provide this on request
|
|
||
| All three deployment scenarios follow a similar pipeline execution pattern, with differences in where authentication happens and where data resides: | ||
|
|
||
| ### Standard Data Flow Steps |
There was a problem hiding this comment.
This definitely needs a diagram
There was a problem hiding this comment.
Agreed - we might even have one laying around somewhere
|
|
||
| **SaaS**: Metadata is stored in ZenML infrastructure. Your ML data and compute remain in your infrastructure. | ||
|
|
||
| **Hybrid**: Metadata and control plane are split — authentication/RBAC happens at ZenML control plane, but all run metadata, artifacts, and compute stay in your infrastructure. |
There was a problem hiding this comment.
I thnk the authentication bit is the most important here and isnt really elaborated but maybe it is later?
There was a problem hiding this comment.
What more would you like to know about this at this stage?
|
|
||
| You control this access by configuring appropriate cloud IAM permissions. | ||
|
|
||
| ## Getting Started |
There was a problem hiding this comment.
IMO super strnage to have this whole section here...
There was a problem hiding this comment.
the whole section ? maybe we dont need the example pipeline - butt i like how it shows how quickly youi're ready
There was a problem hiding this comment.
Hmm really? its in the dashboard already when you sign up
There was a problem hiding this comment.
well somebody in the docs here wants to know what complexity awaits them - "Is it worth my time?"
There was a problem hiding this comment.
in my experience these are the questions we get very early on
Co-authored-by: Hamza Tahir <hamza@zenml.io>
… docs/better-pro-docs
|
Images automagically compressed by Calibre's image-actions ✨ Compression reduced images by 34%, saving 132.21 KB.
383 images did not require optimisation. Update required: Update image-actions configuration to the latest version before 1/1/21. See README for instructions. |
stefannica
left a comment
stefannica
left a comment
There was a problem hiding this comment.
first round of reviews, more to follow...
| #### ZenML Pro Client Artifacts | ||
|
|
||
| If you're planning on running containerized ZenML pipelines, or using other containerization related ZenML features, you'll also need to access the public ZenML client container image located [in Docker Hub at `zenmldocker/zenml`](https://hub.docker.com/r/zenmldocker/zenml). This isn't a problem unless you're deploying ZenML Pro in an air-gapped environment, in which case you'll also have to copy the client container image into your own container registry. You'll also have to configure your code to use the correct base container registry via DockerSettings (see the [DockerSettings documentation](https://docs.zenml.io/how-to/customize-docker-builds) for more information). | ||
| If you're planning on running containerized ZenML pipelines, or using other containerization related ZenML features, you'll also need to access the public ZenML client container image located [in Docker Hub at `zenmldocker/zenml`](https://hub.docker.com/r/zenmldocker/zenml). This isn't a problem unless you're deploying ZenML Pro in a Self-hosted environment, in which case you'll also have to copy the client container image into your own container registry. You'll also have to configure your code to use the correct base container registry via DockerSettings (see the [DockerSettings documentation](https://docs.zenml.io/how-to/customize-docker-builds) for more information). |
There was a problem hiding this comment.
| If you're planning on running containerized ZenML pipelines, or using other containerization related ZenML features, you'll also need to access the public ZenML client container image located [in Docker Hub at `zenmldocker/zenml`](https://hub.docker.com/r/zenmldocker/zenml). This isn't a problem unless you're deploying ZenML Pro in a Self-hosted environment, in which case you'll also have to copy the client container image into your own container registry. You'll also have to configure your code to use the correct base container registry via DockerSettings (see the [DockerSettings documentation](https://docs.zenml.io/how-to/customize-docker-builds) for more information). | |
| If you're planning on running containerized ZenML pipelines, or using other containerization related ZenML features, you'll also need to access the public ZenML client container image located [in Docker Hub at `zenmldocker/zenml`](https://hub.docker.com/r/zenmldocker/zenml). This isn't a problem unless you're deploying ZenML Pro in an air-gapped environment, in which case you'll also have to copy the client container image into your own container registry. You'll also have to configure your code to use the correct base container registry via DockerSettings (see the [DockerSettings documentation](https://docs.zenml.io/how-to/customize-docker-builds) for more information). |
The original text was actually correct here.
| Choose **Self-hosted** if you need complete control with no external dependencies. | ||
|
|
||
| **What runs where:** | ||
| - All components: [Your infrastructure](https://docs.zenml.io/stacks) (completely isolated) |
There was a problem hiding this comment.
It doesn't make sense to point to stacks here.
|
|
||
| 1. **Code Execution**: You write code and run pipelines with your client SDK using Python | ||
|
|
||
| 2. **Token Acquisition**: The ZenML client fetches short-lived tokens from your ZenML workspace for: |
There was a problem hiding this comment.
note: this only happens if you use service connectors
|
|
||
| | Component | Location | Purpose | | ||
| |-----------|----------|---------| | ||
| | **ZenML Pro Server** | ZenML Infrastructure | Manages pipeline orchestration and metadata | |
There was a problem hiding this comment.
You are using the term "zenml server" in several places, but it appears in none of the diagrams. You should probably be using "zenml workspace".
|
|
||
| | Deployment Aspect | SaaS | Hybrid SaaS | Self-hosted | | ||
| |-------------------|------|-------------|------------| | ||
| | **ZenML Server** | ZenML infrastructure | Your infrastructure | Your infrastructure | |
There was a problem hiding this comment.
Should this be workspace instead of server ?
|
|
||
| ### 🚀 Production Ready | ||
|
|
||
| - **High availability**: Built-in redundancy for critical components |
There was a problem hiding this comment.
Incorrect. I would say that the workspaces are the critical components, not the control plane or UI, and they are not under our control, so we cannot offer any such guarantees and shouldn't make such claims.
| - **High availability**: Built-in redundancy for critical components | ||
| - **Automatic updates**: Control plane maintained by ZenML | ||
| - **Professional support**: Direct access to ZenML experts | ||
| - **Monitoring included**: Health checks and alerting configured |
There was a problem hiding this comment.
Only partially correct. Health checks and alerting is only partial configured for the control plane. Workspaces are not covered.
|
|
||
| ### Artifact Store Access | ||
|
|
||
| The ZenML dashboard requires read access to your artifact store to display: |
There was a problem hiding this comment.
This should say UI (not dashboard) and it requires access to more than the artifact store (log store, orchestrators; see my other comment)
| - Artifact lineage graphs | ||
| - Step logs and outputs | ||
|
|
||
| You control this access by configuring appropriate cloud IAM permissions. |
There was a problem hiding this comment.
This is misleading. The truth is this: if you give your users permission to access these things, you also implicitly give the UI permission to do so. You could say "you control who can access this information in the UI by configuring appropriate ZenML Pro RBAC permissions.
| ```mermaid | ||
| graph LR | ||
| A[User] -->|1. Login| B[Control Plane<br/>ZenML Infrastructure] | ||
| B -->|2. Auth Token| A | ||
| A -->|3. Access Workspace| C[Workspace<br/>Your Infrastructure] | ||
| C -->|4. Validate Token| B | ||
| B -->|5. Authorization| C | ||
| C -->|6. Execute| D[Your Resources] | ||
| ``` |
There was a problem hiding this comment.
these do not render correctly
Describe changes
I added a section per deployment scenario - https://zenml-io.gitbook.io/alexej/zenml-pro
Pre-requisites
Please ensure you have done the following:
developand the open PR is targetingdevelop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.Types of changes