[BugFix][CI/Build][ROCM] Fix import error and apply assert in appropriate case in test_struct_output_generate

[rasmith]

This PR fixes and import error in test_struct_output_generate which was importing get_num_sms from device_info when it should have been arch_info. There was also an assert that should have been applied only for Qwen3 but was being applied to other models.

All tests now pass.

This fixes issue 27844.

FYI, I found no evidence of OOM during the execution of a successful test. If a test fails, then it looks like leakage can occur which can cause the remaining tests to OOM, but this seems like a separate issue.

============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-8.4.2, pluggy-1.6.0
rootdir: /vllm-upstream
configfile: pyproject.toml
plugins: asyncio-1.2.0, anyio-4.11.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 20 items

tests/v1/entrypoints/llm/test_struct_output_generate.py ................ [ 80%]
.... [100%]

=============================== warnings summary ===============================
:488
:488: DeprecationWarning: builtin type SwigPyPacked has no module attribute

:488
:488: DeprecationWarning: builtin type SwigPyObject has no module attribute

tests/v1/entrypoints/llm/test_struct_output_generate.py::test_structured_output[mistralai/Ministral-8B-Instruct-2410-xgrammar-auto-None]
tests/v1/entrypoints/llm/test_struct_output_generate.py::test_structured_output[mistralai/Ministral-8B-Instruct-2410-guidance-auto-None]
tests/v1/entrypoints/llm/test_struct_output_generate.py::test_structured_output[mistralai/Ministral-8B-Instruct-2410-lm-format-enforcer-auto-None]
tests/v1/entrypoints/llm/test_struct_output_generate.py::test_structured_output[mistralai/Ministral-8B-Instruct-2410-outlines-auto-speculative_config6]
tests/v1/entrypoints/llm/test_struct_output_generate.py::test_structured_output[mistralai/Ministral-8B-Instruct-2410-guidance-auto-speculative_config7]
tests/v1/entrypoints/llm/test_struct_output_generate.py::test_structured_output_auto_mode[mistralai/Ministral-8B-Instruct-2410-auto]
/vllm-upstream/vllm/transformers_utils/tokenizer.py:287: FutureWarning: It is strongly recommended to run mistral models with --tokenizer-mode "mistral" to ensure correct encoding and decoding.
return get_tokenizer(

tests/v1/entrypoints/llm/test_struct_output_generate.py::test_structured_output_with_structural_tag[xgrammar]
/vllm-upstream/tests/v1/entrypoints/llm/test_struct_output_generate.py:899: DeprecationWarning: guided_decoding is deprecated. This will be removed in v0.12.0 or v1.0.0, which ever is soonest. Please use structured_outputs instead.
sampling_params = SamplingParams(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================== 20 passed, 9 warnings in 762.31s (0:12:42) ==================

[gemini-code-assist]

Code Review

This pull request correctly addresses two issues: an import error in rocm_aiter_fa.py and an overly strict assertion in test_struct_output_generate.py. The change in rocm_aiter_fa.py corrects the import path for get_num_sms. The modification in test_struct_output_generate.py makes the assertion for content conditional on the model name, which aligns with the intended testing logic. The changes are well-implemented and resolve the described bugs. I have no further comments.

[micah-wil]

AITER main has get_num_sms in aiter.ops.triton.utils.device_info which is why it is imported as such (https://github.com/ROCm/aiter/tree/main/aiter/ops/triton/utils). Also, in AITER main, arch_info.py was moved to aiter.ops.triton.utils._triton (https://github.com/ROCm/aiter/tree/main/aiter/ops/triton/utils/_triton) and get_num_sms no longer lives there. So it seems like this import will fail depending on what AITER commit you are using

[rasmith]

AITER main has get_num_sms in aiter.ops.triton.utils.device_info which is why it is imported as such (https://github.com/ROCm/aiter/tree/main/aiter/ops/triton/utils). Also, in AITER main, arch_info.py was moved to aiter.ops.triton.utils._triton (https://github.com/ROCm/aiter/tree/main/aiter/ops/triton/utils/_triton) and get_num_sms no longer lives there. So it seems like this import will fail depending on what AITER commit you are using

It doesn't match what vLLM is currently installing. When I looked at what was actually installed, this was the correct import.

[micah-wil]

AITER main has get_num_sms in aiter.ops.triton.utils.device_info which is why it is imported as such (https://github.com/ROCm/aiter/tree/main/aiter/ops/triton/utils). Also, in AITER main, arch_info.py was moved to aiter.ops.triton.utils._triton (https://github.com/ROCm/aiter/tree/main/aiter/ops/triton/utils/_triton) and get_num_sms no longer lives there. So it seems like this import will fail depending on what AITER commit you are using

It doesn't match what vLLM is currently installing. When I looked at what was actually installed, this was the correct import.

Yep the AITER commit in the base dockerfile is slightly old now, just adding the context here for anyone playing around with newer AITER commits.

[zhewenl]

@rasmith : The OOM issue can be reproducible if you run pytest -v -s v1/entrypoints to test all the tests (same behavior in what we do in CI)

https://github.com/vllm-project/vllm/blob/main/.buildkite/test-amd.yaml#L328

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @rasmith.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @rasmith.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[tjtanaa]

@rasmith Can you rebase this as well?