[Feature] Could you please share softmoe config folder?

[giangdip2410]

Describe the feature

Hi @Adlith, thank you for your great work. Could you please share with me softmoe config?

Will you implement it?

• I would like to implement this feature and create a PR!

[Adlith]

Thank you for your interest! You can follow the Soft MoE implementation by Lucidrains.
We also plan to integrate this into our repo, but it might take a bit more time. Feel free to reach out if you need any help!