40 5 34

BlinkDL

https://www.rwkv.com/

AI & ML interests

RWKV is all you need

Recent Activity

updated a model 1 day ago

BlinkDL/temp-latest-training-models

updated a model 5 days ago

BlinkDL/rwkv-7-world

updated a model 11 days ago

BlinkDL/rwkv-8-pile

View all activity

Organizations

Posts 3

Post

7728

RWKV-7 "Goose" 0.4B trained w/ ctx4k automatically extrapolates to ctx32k+, and perfectly solves NIAH ctx16k 🤯 100% RNN and attention-free. Only trained on the Pile. No finetuning. Replicable training runs. tested by our community: https://github.com/Jellyfish042/LongMamba

Post

9992

RWKV-6-world-v3 (+3.1T tokens) is our best multilingual 7B model as of now: BlinkDL/rwkv-6-world

It's 100% RNN and attention-free. MMLU 54.2% (previous world-v2.1 = 47.9%. note: without eval-boosting tricks such as annealing).

RWKV-7-world-v4 soon :)

View all Posts