Loading your model checkpoints

[MartinBenfeghoul]

Hi,

Firstly, thank you for opensourcing your code and weights.

I am however, having issues using your checkpoints. How would you recommend loading and evaluating your own model checkpoints? For example for this one, I keep getting the following error when trying to load it in using transformers (4.56.1, as installed as a dependency of flash-linear-attention-0.3.2):

ValueError: The checkpoint you are trying to load has model type `liger_gla` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

When trying to load them into your LigerGLAForCausalLM class directly I get the following error, just on the import of said class:

File ~/Linearization/liger/models/liger_gla/__init__.py:7
      4 from liger.models.liger_gla.modeling_liger_gla import LigerGLAForCausalLM, LigerGLAModel
      6 AutoConfig.register(LigerGLAConfig.model_type, LigerGLAConfig)
----> 7 AutoModel.register(LigerGLAConfig, LigerGLAModel)
      8 AutoModelForCausalLM.register(LigerGLAConfig, LigerGLAForCausalLM)
     11 __all__ = ['LigerGLAConfig', 'LigerGLAForCausalLM', 'LigerGLAModel']

File ~/miniconda3/envs/liger/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py:624, in _BaseAutoModelClass.register(cls, config_class, model_class, exist_ok)
    614 """
    615 Register a new model for this class.
    616 
   (...)    621         The model to register.
    622 """
    623 if hasattr(model_class, "config_class") and model_class.config_class.__name__ != config_class.__name__:
--> 624     raise ValueError(
    625         "The model class you are passing has a `config_class` attribute that is not consistent with the "
    626         f"config class you passed (model has {model_class.config_class} and you passed {config_class}. Fix "
    627         "one of those so they match!"
    628     )
    629 cls._model_mapping.register(config_class, model_class, exist_ok=exist_ok)

ValueError: The model class you are passing has a `config_class` attribute that is not consistent with the config class you passed (model has <class 'transformers.models.llama.configuration_llama.LlamaConfig'> and you passed <class 'liger.models.liger_gla.configuration_liger_gla.LigerGLAConfig'>. Fix one of those so they match!

This appears to be an issue with the transformers version. To this end, could you please provide specific versions for all requirements, ./third_party/ submodules, and the Python version you used?