Add RWKV v6 1.6B files for ncnn backend
Browse filesSigned-off-by: Molly Sophia <[email protected]>
ncnn/RWKV-x060-World-1B6/RWKV-x060-World-1B6-v2.1-20240328-ctx4096-ncnn.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6800486c730297d796df09bb0edc69f6aa65547e221cd5299929473886d3a84c
|
| 3 |
+
size 3115731308
|
ncnn/RWKV-x060-World-1B6/RWKV-x060-World-1B6-v2.1-20240328-ctx4096-ncnn.config
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version: 6
|
| 2 |
+
act_dtype: fp32
|
| 3 |
+
weight_dtype: fp16
|
| 4 |
+
head_size: 32
|
| 5 |
+
n_layer: 24
|
| 6 |
+
n_embd: 2048
|
| 7 |
+
n_att: 2048
|
| 8 |
+
n_ffn: 7168
|
| 9 |
+
ncnn_impl_version: 2
|
ncnn/RWKV-x060-World-1B6/RWKV-x060-World-1B6-v2.1-20240328-ctx4096-ncnn.param
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|