Set Block Decoding

[davidmrau]

Feature request

Adding Set Block Decoding for Training and inference.

https://huggingface.co/papers/2509.04185

Motivation

Speeding up generation time with minimal additional fine-tuning.

Your contribution

Could implement a first draft.

[Rocketknight1]

cc @gante!

[itaigat]

thanks @davidmrau! im happy to support this. however, we have an upcoming improved version with better speedups and vllm implementation, worth waiting a bit

[davidmrau]

Sounds great!