DALI backend on Jetson AGX

[jilongliao]

Hi,

I'm planning to run tritonserver on Jetson. I was able to pull nvcr.io/nvidia/tritonserver:25.05-py3-igpu and run the server on jetson. However, i realize that the DALI backend is not available in the igpu image.

docker run --runtime=nvidia --network=host --rm -p 8000:8000 -p 8001:8001 -p 8002:8002 -v /mnt/data/models:/models -e YOLO_VERBOSE=false -it nvcr.io/nvidia/tritonserver:25.05-py3-igpu ls /opt/tritonserver/backends/
WARNING: Published ports are discarded when using host network mode

=============================
== Triton Inference Server ==
=============================

NVIDIA Release 25.05 (build 170551412)
Triton Server Version 2.58.0

Copyright (c) 2018-2025, NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

GOVERNING TERMS: The software and materials are governed by the NVIDIA Software License Agreement
(found at https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-software-license-agreement/)
and the Product-Specific Terms for NVIDIA AI Products
(found at https://www.nvidia.com/en-us/agreements/enterprise-software/product-specific-terms-for-ai-products/).

fil  identity  onnxruntime  python  pytorch  tensorrt

I am not sure how to make DALI backend on Jetson. Can someone help?

[szalpal]

Hi @jilongliao !

To obtain Jetson build for Triton with DALI Backend, you'd need the following:

1. Build DALI for Jetson from source. The instruction is here. This process will create a DALI wheel for Jetson platform.
2. Pull Triton docker image for Jetson.
3. Use Triton Jetson image to add DALI Backend, together with DALI wheel obtained in step 1. This Dockerfile outlines, how this can be achieved (you can for example copy this one, call it Dockerfile.jetson and this Dockerfile will build a new tritonserver docker image for you, with DALI Backend inside). You'd need to swap the BASE_IMAGE in this line to the Jetson version. Then, you need to build DALI Backend (and deploy it) inside the Dockerfile.jetson. I believe the easiest approach would be to install DALI Jetson wheel inside the Dockerfile and then set -D TRITON_DALI_SKIP_DOWNLOAD=ON in the cmake command that runs DALI Backend build. This approach is outlined here.

I'm aware that these are really manual steps, however we never approached building DALI Backend for Jetson in past. Should you like to create a contribution to DALI Backend (e.g. docker/Dockerfile.jetson which creates the Triton image), I'd be very grateful :)

[jilongliao]

@szalpal thanks for the pointers. I'm able to build!!! I'd love to contribute and send a PR here. this is cool!

[jilongliao]

@szalpal i have the commit available, but you probably need to give me permission to push to a branch that i can send the PR.

ERROR: Permission to triton-inference-server/dali_backend.git denied to jilongliao.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

[JanuszL]

Hi @jilongliao,

@szalpal i have the commit available, but you probably need to give me permission to push to a branch that i can send the PR.

ERROR: Permission to triton-inference-server/dali_backend.git denied to jilongliao.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Please fork the main repo, create branch on your private fork and create a PR between your fork and the main repo.

[jilongliao]

Hi @jilongliao,

@szalpal i have the commit available, but you probably need to give me permission to push to a branch that i can send the PR.
ERROR: Permission to triton-inference-server/dali_backend.git denied to jilongliao.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

Please fork the main repo, create branch on your private fork and create a PR between your fork and the main repo.

ACK. will do!

[jilongliao]

PR here: #272