Tiny non-Transformers

FlameF0X 's Collections

Benchmarks

TinyMoE, Dense & More

MCP

Reinforcement Learning

updated 2 days ago

Any model that is not pure transformers, so both RWKV, Mamba like models and hybrid like Jamba will be showed. MODELS OF THE SAME FAMILY DONT GET NOTE

Upvote

state-spaces/mamba-130m-hf

Text Generation • 0.1B • Updated Mar 6, 2024 • 308k • 76

Note Type: SSM
BlinkDL/rwkv7-g1

Text Generation • Updated 7 days ago • 17k • 229

Note Type: RNN It has the weigh of the RWKV G1 0.1B 0.4B and 1.5B
BlinkDL/rwkv-4-pile-1b5

Text Generation • Updated Mar 13, 2023 • 770 • 27
RWKV/rwkv-4-169m-pile

Text Generation • Updated May 15, 2023 • 9.74k • 9
FWKV/Myosotis-1-base

Text Generation • 0.1B • Updated 3 days ago • 2.09k • 21

Note Type: RNN
FWKV/FWKV-ROSA

Text Generation • 57M • Updated Aug 7 • 59 • 3
FWKV/FWKV-50M

Text Generation • 50.6M • Updated May 19 • 27
FWKV/FWKV-29M

Text Generation • 28.9M • Updated May 14 • 251
basically-ai/Pebble-10M

Text Generation • 11.1M • Updated 27 days ago • 643 • 11

Note Type: Hybrid between Transformer and Mamba2
basically-ai/Pebble-25M-Chat

Text Generation • 24.5M • Updated 24 days ago • 589 • 7
basically-ai/Pebble-10M-Chat

Text Generation • 11.1M • Updated 25 days ago • 1.55k • 13
LiquidAI/LFM2.5-230M

Text Generation • 0.2B • Updated Aug 19 • 82.9k • 299

Note Type; Hybrid
LiquidAI/LFM2.5-350M

Text Generation • 0.4B • Updated Aug 5 • 78k • 426
LiquidAI/LFM2.5-1.2B-Instruct

Text Generation • 1B • Updated Aug 24 • 112k • 673
sz14/cRia-LM-75M

Text Generation • 75.7M • Updated 8 days ago • 6.27k • 10

Note Type: Weird hybrid or something? It has no KV cache so i think it counts.
fromziro/ZeroS-Linear-50M

49.6M • Updated 12 days ago • 404 • 3

Note Type: RNN
User01110/Linear-50M-exp

49.6M • Updated 21 days ago • 346 • 3

Upvote