Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/book/component-guide/artifact-stores/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ Out of the box, ZenML comes with a `local` artifact store already part of the de
| [Amazon S3](s3.md) | `s3` | `s3` | `s3://` | Uses AWS S3 as an object store backend |
| [Google Cloud Storage](gcp.md) | `gcp` | `gcp` | `gs://` | Uses Google Cloud Storage as an object store backend |
| [Azure](azure.md) | `azure` | `azure` | `abfs://`, `az://` | Uses Azure Blob Storage as an object store backend |
| [Alibaba Cloud OSS](alibaba-oss.md) | `s3` | `s3` | `s3://` | Uses S3 integration to connect to Alibaba Cloud OSS |
| [MinIO](minio.md) | `s3` | `s3` | `s3://` | Uses S3 integration to connect to self-hosted MinIO |
| [Custom Implementation](custom.md) | _custom_ | | _custom_ | Extend the Artifact Store abstraction and provide your own implementation |

If you would like to see the available flavors of Artifact Stores, you can use the command:
Expand Down
92 changes: 92 additions & 0 deletions docs/book/component-guide/artifact-stores/alibaba-oss.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
description: Storing artifacts in Alibaba Cloud Object Storage Service (OSS).
---

# Alibaba Cloud OSS

Alibaba Cloud Object Storage Service (OSS) is an S3-compatible object storage service. Since OSS provides an S3-compatible API, you can use ZenML's S3 Artifact Store integration to connect to Alibaba Cloud OSS.

### When would you want to use it?

You should use the Alibaba Cloud OSS Artifact Store when you want to store your ZenML artifacts in Alibaba Cloud infrastructure, particularly if:

* You're already using Alibaba Cloud services
* You need to store artifacts in specific geographic regions served by Alibaba Cloud
* You want to leverage Alibaba Cloud's pricing and performance characteristics

### How do you deploy it?

Since Alibaba Cloud OSS is S3-compatible, you'll use the S3 integration. First, install it:

```shell
zenml integration install s3 -y
```

You'll also need to create an OSS bucket and obtain your access credentials from the Alibaba Cloud console.

### How do you configure it?

To use Alibaba Cloud OSS with ZenML, you need to configure the S3 Artifact Store with specific settings for OSS compatibility:

{% tabs %}
{% tab title="Using a ZenML Secret (recommended)" %}

First, create a ZenML secret with your Alibaba Cloud credentials:

```shell
zenml secret create alibaba_secret \
--aws_access_key_id='<YOUR_ALIBABA_ACCESS_KEY_ID>' \
--aws_secret_access_key='<YOUR_ALIBABA_SECRET_ACCESS_KEY>'
```

Then register the artifact store with the required OSS configuration:

```shell
zenml artifact-store register alibaba_store -f s3 \
--path='s3://your-bucket-name' \
--authentication_secret=alibaba_secret \
--client_kwargs='{"endpoint_url": "https://oss-<region>.aliyuncs.com"}' \
--config_kwargs='{"signature_version": "s3", "s3": {"addressing_style": "virtual"}}'
```
{% endtab %}

{% tab title="Inline credentials" %}

You can also provide credentials directly (not recommended for production):

```shell
zenml artifact-store register alibaba_store -f s3 \
--path='s3://your-bucket-name' \
--aws_access_key_id='<YOUR_ALIBABA_ACCESS_KEY_ID>' \
--aws_secret_access_key='<YOUR_ALIBABA_SECRET_ACCESS_KEY>' \
--client_kwargs='{"endpoint_url": "https://oss-<region>.aliyuncs.com"}' \
--config_kwargs='{"signature_version": "s3", "s3": {"addressing_style": "virtual"}}'
```
{% endtab %}
{% endtabs %}

{% hint style="warning" %}
**Important:** When using Alibaba Cloud OSS, you must set the following `config_kwargs`:

```json
{"signature_version": "s3", "s3": {"addressing_style": "virtual"}}
```

This is required for proper compatibility with Alibaba Cloud OSS's S3 API implementation.
{% endhint %}

Replace `<region>` with your OSS region (e.g., `eu-central-1`, `cn-hangzhou`, `ap-southeast-1`). You can find the list of available regions and their endpoints in the [Alibaba Cloud OSS documentation](https://www.alibabacloud.com/help/en/oss/user-guide/regions-and-endpoints).

Finally, add the artifact store to your stack:

```shell
zenml stack register custom_stack -a alibaba_store ... --set
```

### How do you use it?

Using the Alibaba Cloud OSS Artifact Store is no different from [using any other flavor of Artifact Store](./#how-to-use-it). ZenML handles the S3-compatible API translation automatically.

For more details on the S3 Artifact Store configuration options, refer to the [S3 Artifact Store documentation](s3.md).

<figure><img src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" alt="ZenML Scarf"><figcaption></figcaption></figure>
89 changes: 89 additions & 0 deletions docs/book/component-guide/artifact-stores/minio.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
---
description: Storing artifacts in MinIO object storage.
---

# MinIO

[MinIO](https://min.io/) is a high-performance, S3-compatible object storage system. Since MinIO provides a fully S3-compatible API, you can use ZenML's S3 Artifact Store integration to connect to MinIO.

### When would you want to use it?

You should use the MinIO Artifact Store when:

* You want to run a self-hosted object storage solution
* You need S3-compatible storage in your own data center or private cloud
* You're running ZenML in a Kubernetes environment and want co-located storage
* You want to avoid cloud vendor lock-in while maintaining S3 API compatibility

### How do you deploy it?

Since MinIO is S3-compatible, you'll use the S3 integration. First, install it:

```shell
zenml integration install s3 -y
```

You'll also need a running MinIO instance. MinIO can be deployed in various ways:

* **Docker**: `docker run -p 9000:9000 -p 9001:9001 minio/minio server /data --console-address ":9001"`
* **Kubernetes**: Using the [MinIO Operator](https://min.io/docs/minio/kubernetes/upstream/)
* **Binary**: Download from [MinIO's website](https://min.io/download)

### How do you configure it?

To use MinIO with ZenML, configure the S3 Artifact Store with your MinIO endpoint:

{% tabs %}
{% tab title="Using a ZenML Secret (recommended)" %}

First, create a ZenML secret with your MinIO credentials:

```shell
zenml secret create minio_secret \
--aws_access_key_id='<YOUR_MINIO_ACCESS_KEY>' \
--aws_secret_access_key='<YOUR_MINIO_SECRET_KEY>'
```

Then register the artifact store:

```shell
zenml artifact-store register minio_store -f s3 \
--path='s3://your-bucket-name' \
--authentication_secret=minio_secret \
--client_kwargs='{"endpoint_url": "http://minio.example.com:9000"}'
```
{% endtab %}

{% tab title="Inline credentials" %}

You can also provide credentials directly (not recommended for production):

```shell
zenml artifact-store register minio_store -f s3 \
--path='s3://your-bucket-name' \
--aws_access_key_id='<YOUR_MINIO_ACCESS_KEY>' \
--aws_secret_access_key='<YOUR_MINIO_SECRET_KEY>' \
--client_kwargs='{"endpoint_url": "http://minio.example.com:9000"}'
```
{% endtab %}
{% endtabs %}

Replace `http://minio.example.com:9000` with your actual MinIO endpoint. If you're running MinIO locally for development, this might be `http://localhost:9000`.

{% hint style="info" %}
If your MinIO instance uses HTTPS with a self-signed certificate, you may need to configure SSL verification. Consult the [S3 Artifact Store documentation](s3.md#advanced-configuration) for advanced configuration options.
{% endhint %}

Finally, add the artifact store to your stack:

```shell
zenml stack register custom_stack -a minio_store ... --set
```

### How do you use it?

Using the MinIO Artifact Store is no different from [using any other flavor of Artifact Store](./#how-to-use-it). ZenML handles the S3-compatible API translation automatically.

For more details on the S3 Artifact Store configuration options, refer to the [S3 Artifact Store documentation](s3.md).

<figure><img src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" alt="ZenML Scarf"><figcaption></figcaption></figure>
2 changes: 2 additions & 0 deletions docs/book/component-guide/toc.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@
* [Amazon Simple Cloud Storage (S3)](artifact-stores/s3.md)
* [Google Cloud Storage (GCS)](artifact-stores/gcp.md)
* [Azure Blob Storage](artifact-stores/azure.md)
* [Alibaba Cloud OSS](artifact-stores/alibaba-oss.md)
* [MinIO](artifact-stores/minio.md)
* [Develop a custom artifact store](artifact-stores/custom.md)
* [Container Registries](container-registries/README.md)
* [Default Container Registry](container-registries/default.md)
Expand Down
Loading