-
Notifications
You must be signed in to change notification settings - Fork 29.8k
Open
Labels
Feature requestRequest for a new featureRequest for a new feature
Description
Feature request
Muon: https://github.com/KellerJordan/Muon
Flash-Muon: https://github.com/nil0x9/flash-muon
Paper: https://arxiv.org/pdf/2502.16982
Motivation
An effective optimizer method to further accelerate LLM training. The Muon team has recently proven the importance of the muon optimizer in the LLM models they released.

Your contribution
I want to add the transformers library to this optimization.
Metadata
Metadata
Assignees
Labels
Feature requestRequest for a new featureRequest for a new feature