Skip to content

Missing support for liger_kernel patches for recent models #5085

@sfc-gh-mkrubinski

Description

@sfc-gh-mkrubinski

Describe the feature
Update the list of supported liger kernels to include patches for newer models (Qwen3, Gemma-3, ...)

Paste any useful information
The list of liger_kernels supported by ms-swift

from liger_kernel.transformers import (apply_liger_kernel_to_llama, apply_liger_kernel_to_mistral,
apply_liger_kernel_to_mixtral, apply_liger_kernel_to_gemma,
apply_liger_kernel_to_qwen2, apply_liger_kernel_to_qwen3,
apply_liger_kernel_to_qwen2_vl, apply_liger_kernel_to_qwen2_5_vl,
apply_liger_kernel_to_phi3, apply_liger_kernel_to_mllama)
does not include the more recent ones released by Liger-Kernel.

Missing ones include Qwen3: liger_kernel.transformers.apply_liger_kernel_to_qwen3 and Gemma-3: liger_kernel.transformers.apply_liger_kernel_to_gemma3.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions