Releases: lucidrains/native-sparse-attention-pytorch
Releases · lucidrains/native-sparse-attention-pytorch
0.0.14
wire up flex attention for sliding windows
0.0.12
allow for the strategy combine "mlp" to be customized as well, but th…
0.0.11
small test for customizable compress mlp
0.0.9
give the compress mlp some depth, then allow it to be customizable by…
0.0.8
Full Changelog: 0.0.7...0.0.8
0.0.7
redo get_at with gather, but keep around the ein notation for readabi…
0.0.6
fix after changing importance score to compressed attention values
0.0.5
coordinate descent was unstable, just use a one hot straight through …
0.0.4
Full Changelog: 0.0.3...0.0.4
0.0.3
fix an issue with mask, make sure it converges for enwik8