a small bug in qwen3_moe

[shiwanghua]

System Info

transformers/src/transformers/models/qwen3_moe/modular_qwen3_moe.py

Line 106 in cbb290e

idx, top_x = torch.where(expert_mask[expert_idx].squeeze(0))

The names of the first and second variables should be exchanged:

top_x, idx = torch.where(expert_mask[expert_idx].squeeze(0))

In the subsequent code, replace the topk with idx and vice versa.

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

wrong name meaning

Expected behavior

change the codes

[shiwanghua]

This line of code best should be modified to:

current_state = hidden_states[top_x, :]

By deepseek:

[Rocketknight1]

cc @ArthurZucker for MoE!