[CI Failure][AMD] Encoder-Decoder Models Fail on AMD CI

[zhewenl]

Name of failing test

entrypoints/openai/test_transcription_validation.py::test_basic_audio[openai/whisper-large-v3-turbo]

Basic information

• Flaky test
• Can reproduce locally
• Caused by external libraries (e.g. bug in transformers)

🧪 Describe the failing test

Summary

Encoder-decoder models (Whisper, T5, BART, vision-language models) fail on AMD ROCm with NotImplementedError because all ROCm-specific attention backends only support decoder-only models.

Therefore, all the tests using these models will fail on AMD CI(example), mainly in Entrypoints Integration Test (API Server) and Entrypoints Integration Test (Pooling).

Affected Models

• Speech-to-Text: Whisper family (openai/whisper-large-v3-turbo, mistralai/Voxtral-Mini-3B-2507)
• Vision-Language: microsoft/Phi-3.5-vision-instruct
• Translation: T5, BART, MarianMT models
• Any model with cross-attention: Encoder-decoder architectures

Failure Mode

Error:

NotImplementedError: Encoder self-attention and encoder/decoder cross-attention
are not implemented for TritonAttentionImpl

Root Cause: All ROCm-Specific Backends are Decoder-Only

Backend	Location
TritonAttentionBackend(default)	vllm/v1/attention/backends/triton_attn.py
RocmAttentionBackend	vllm/v1/attention/backends/rocm_attn.py
RocmAiterFABackend	vllm/v1/attention/backends/rocm_aiter_fa.py
RocmAiterMLABackend	vllm/v1/attention/backends/mla/rocm_aiter_mla.py

eg. TritonAttentionImpl:

if attn_type != AttentionType.DECODER:
    raise NotImplementedError(
        "Encoder self-attention and "
        "encoder/decoder cross-attention "
        "are not implemented for "
        "TritonAttentionImpl"
    )

Purposal

1. [short-term] Mark encoder-decoder tests with @pytest.encoder_decoder, and skip them when using AMD.
2. [long-term] Support flash_attn on ROCM, which is missing here - it will also require supporting kv bindings here.

📝 History of failing test

https://buildkite.com/vllm/ci/builds/35670#019a04ed-a2f7-4e7e-85ad-4e2118f76898

CC List.

@DarkLight1337 @LucasWilkinson @simon-mo @yeqcharlotte @Alexei-V-Ivanov-AMD @gshtras

[github-actions]

CC @hongxiayang @tjtanaa @vllmellm for ROCm-related issue

[Alexei-V-Ivanov-AMD]

@zhewenl Thank you for raising this issue.
We have also identified this as one of the most important features.

[zhewenl]

#27605: this PR turned off testing encoder-decoder models in Example Tests

[ugur-stackav]

Is there any plan to support encoder-decoder models in AMD? Also on vllm V0 engine it works but I am seeing memory related issues with different batch sizes there.

[zhewenl]

Attempting to skip tests related encoder-deocder models in API Server/Pooling tests inhttps://github.com//pull/28156

@maleksan85 could you help check remaining failing tests in CI? I think PyTorch Fullgraph Smoke Test has some tests related to encoder

[coderfeli]

@zhewenl fixed in #28346