Efficient encoder-decoder architecture for small language models (≤1B parameters) with cross-architecture knowledge distillation and vision-language capabilities

使用Decoder-only的Transformer进行时序预测，包含SwiGLU和RoPE(Rotary Positional Embedding)，Time series prediction using Decoder-only Transformer, Including SwiGLU and RoPE(Rotary Positional Embedding)

🔍 Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment

Code for paper "Modality Plug-and-Play: Elastic Modality Adaptation in Multimodal LLMs for Embodied AI"

ViAG: A Novel Framework for Fine-tuning Answer Generation models ultilizing Encoder-Decoder and Decoder-only Transformers's architecture

A mini version of GPT implemented on shakespear using BPE

This repository contains the implementation and experiments for comparing gradual growth methods, specifically the G_stack approach, with naive models trained from scratch. The project focuses on addressing catastrophic forgetting and improving model performance in continuous learning scenarios.

in dev ...

Auto regressive text generation application using decoder transformer

Decoder-only transformer, simplest character-level tokenization, training and text generation.

🧸 A fully custom GPT-style language model built from scratch using PyTorch and trained on Winnie-the-Pooh! Explored the core mechanics of self-attention, autoregressive text generation, and modular model training, all without relying on any external libraries.

Decoder-only transfomer model for answering short questions using causal self-attention.

A decoder only approach for image reconstruction inspired by adversarial machine learning implemented in keras/tensorflow2