State Space Models

state-spaces's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Articles

tridao 

authored a paper 4 months ago

M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models

Paper • 2504.10449 • Published Apr 14 • 14

tridao 

authored a paper 9 months ago

RedPajama: an Open Dataset for Training Large Language Models

Paper • 2411.12372 • Published Nov 19, 2024 • 57

ArthurZ 

posted an update 9 months ago

view post

Post

4749

Native tensor parallel has landed in transformers!!! https://github.com/huggingface/transformers/pull/34184 thanks a lot to the torch team for their support!

Contributions are welcome to support more models! 🔥

🔥

14

14

❤️

5

5

🤯

3

3

🤝

3

3

+

Reply

tridao 

authored a paper 12 months ago

The Mamba in the Llama: Distilling and Accelerating Hybrid Models

Paper • 2408.15237 • Published Aug 27, 2024 • 43

mamba-checkpoints 

authored a paper about 1 year ago

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

Paper • 2405.21060 • Published May 31, 2024 • 68

albertgu 

authored a paper about 1 year ago

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

Paper • 2405.21060 • Published May 31, 2024 • 68

mamba-checkpoints 

updated a model about 1 year ago

state-spaces/mamba-2.8b

3B • Updated May 11, 2024 • 12.9k • 148

ArthurZ 

authored a paper over 1 year ago

RecurrentGemma: Moving Past Transformers for Efficient Open Language Models

Paper • 2404.07839 • Published Apr 11, 2024 • 48

albertgu 

authored a paper over 1 year ago

Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling

Paper • 2403.03234 • Published Mar 5, 2024 • 15

tridao 

authored a paper over 1 year ago

Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling

Paper • 2403.03234 • Published Mar 5, 2024 • 15

ArthurZ 

posted an update over 1 year ago

view post

Post

mamba is now available in transformers. Thanks to @tridao and @albertgu for this brilliant model! 🚀 and the amazing mamba-ssm kernels powering this!
Checkout the collection here:
state-spaces/transformers-compatible-mamba-65e7b40ab87e5297e45ae406

• 
• 
• 
• 5 replies

·

🤝

10

10

❤️

7

7

+

Reply

ArthurZ 

updated 6 models over 1 year ago

state-spaces/mamba-2.8b-hf

Text Generation • 3B • Updated Mar 6, 2024 • 4.23k • 110

state-spaces/mamba-1.4b-hf

Text Generation • 1B • Updated Mar 6, 2024 • 6.58k • 14

state-spaces/mamba-790m-hf

Text Generation • 0.8B • Updated Mar 6, 2024 • 4.15k • 4

state-spaces/mamba-370m-hf

Text Generation • 0.4B • Updated Mar 6, 2024 • 2.53k • 14

state-spaces/mamba-130m-hf

Text Generation • 0.1B • Updated Mar 6, 2024 • 134k • 59

state-spaces/mamba-390m-hf

Updated Mar 6, 2024

tridao 

authored a paper over 1 year ago

StarCoder 2 and The Stack v2: The Next Generation

Paper • 2402.19173 • Published Feb 29, 2024 • 147

ArthurZ 

authored a paper over 1 year ago

StarCoder 2 and The Stack v2: The Next Generation

Paper • 2402.19173 • Published Feb 29, 2024 • 147

albertgu 

authored a paper over 1 year ago

Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models

Paper • 2402.19427 • Published Feb 29, 2024 • 57