Add CUDA 12.8 support for RTX 5090 / Blackwell GPUs

[Curtisflo]

Pull Request: Add CUDA 12.8 Support for RTX 5090 / Blackwell GPUs

Summary

This PR adds support for NVIDIA's RTX 5090 and Blackwell architecture GPUs (sm_120 compute capability) while maintaining full backward compatibility with existing CUDA 12.1 configurations.

Problem

The RTX 5090 uses the new Blackwell architecture with CUDA compute capability sm_120. Current stable PyTorch releases with CUDA 12.1 do not support sm_120, causing the error:

CUDA error: no kernel image is available for execution on the device

Solution

PyTorch 2.8.0 with CUDA 12.8 is the first stable release to include sm_120 support. This PR provides an additional configuration option for users with Blackwell GPUs while keeping the existing CUDA 12.1 setup as the default.

Changes

New Files Added

1. Dockerfile.cu128 - Alternative Dockerfile with CUDA 12.8 support

   • Uses nvidia/cuda:12.8.1-runtime-ubuntu22.04 base image
   • Installs PyTorch 2.8.0+cu128 before other dependencies to prevent downgrade
   • Installs chatterbox with --no-deps to maintain PyTorch version

2. docker-compose-cu128.yml - Docker Compose configuration for CUDA 12.8

   • Uses Dockerfile.cu128
   • Otherwise identical to standard docker-compose.yml

3. requirements-nvidia-cu128.txt - CUDA 12.8 Python requirements

   • Specifies PyTorch 2.8.0 from cu128 wheel index
   • Documents compatibility with Blackwell architecture

4. README_CUDA128.md - Complete setup guide for CUDA 12.8

   • Quick start instructions
   • Verification steps
   • Troubleshooting guide
   • Compatibility matrix

5. README_PR_CHANGES.md - Suggested additions to main README

Files Modified

None - This PR is 100% backward compatible and adds files only.

Backward Compatibility

✅ Fully backward compatible - No existing files are modified:

• Default Dockerfile remains unchanged (CUDA 12.1)
• Default docker-compose.yml remains unchanged
• All existing requirements files unchanged
• Existing users are completely unaffected

Usage

For RTX 5090 / Blackwell Users

# Using Docker Compose (Recommended)
docker compose -f docker-compose-cu128.yml up -d

# Or manual pip installation
pip install -r requirements-nvidia-cu128.txt
pip install --no-deps git+https://github.com/devnen/chatterbox.git

For All Other Users

Continue using the standard setup - no changes needed:

docker compose up -d
# or
pip install -r requirements-nvidia.txt

Verification

Users can verify sm_120 support with:

docker exec <container> python -c "import torch; print(torch.cuda.get_arch_list())"

Expected output should include: ['sm_70', 'sm_75', 'sm_80', 'sm_86', 'sm_90', 'sm_100', 'sm_120']

Benefits

1. RTX 5090 Support - Enables use of latest NVIDIA hardware
2. Future-Proof - Supports all future Blackwell-based GPUs
3. Latest PyTorch - Users benefit from PyTorch 2.8.0 improvements
4. No Breaking Changes - Existing users unaffected
5. Clear Documentation - Comprehensive setup guide included

Testing

Tested on:

• RTX 5090 with CUDA 12.8.1 drivers (version 570.133.07)
• PyTorch 2.8.0+cu128
• Ubuntu 22.04 host system
• Docker 27.x with NVIDIA Container Toolkit

Verified:

• ✅ Model loads successfully on GPU
• ✅ CUDA kernel execution works (no sm_120 errors)
• ✅ All dependencies install without conflicts
• ✅ Server runs and generates audio correctly
• ✅ Web UI accessible and functional

Compatibility Matrix

GPU Generation	Architecture	Compute Capability	Configuration	PyTorch
RTX 5090 / Blackwell	Blackwell	sm_120	docker-compose-cu128.yml	2.8.0+cu128
RTX 4090 / Ada	Ada Lovelace	sm_90	docker-compose.yml	2.5.1+cu121
RTX 3090 / Ampere	Ampere	sm_86	docker-compose.yml	2.5.1+cu121
RTX 20xx / Turing	Turing	sm_75	docker-compose.yml	2.5.1+cu121

Documentation Updates

Optional README additions provided in README_PR_CHANGES.md:

• Brief mention of CUDA 12.8 option in installation section
• Link to detailed README_CUDA128.md guide
• Note in Docker section about cu128 variant

These can be applied or modified as preferred by the maintainer.

Dependencies

All new dependencies are identical to existing ones, just different versions:

• PyTorch: 2.8.0 (vs 2.5.1)
• CUDA: 12.8 (vs 12.1)
• Base image: cuda:12.8.1-runtime (vs cuda:12.6.2-runtime)

Open Question for Maintainer

1. README updates: Apply suggested changes or keep as separate file?

Checklist

• Code follows project style
• Backward compatible (no breaking changes)
• Documentation provided (README_CUDA128.md)
• Tested on target hardware (RTX 5090)
• All new files included
• Docker configurations tested

Related Links

• PyTorch 2.8.0 Release Notes
• CUDA 12.8 Documentation
• RTX 5090 Specifications