Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,10 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
+
This is a tiny Jamba model used for development, debugging and experimentation over the Jamba architecture.
|
| 6 |
+
|
| 7 |
+
It has 319M parameters (instead of 52B in [Jamba 1.5 Mini](https://huggingface.co/ai21labs/AI21-Jamba-1.5-Mini) (and [Jamba v0.1](https://huggingface.co/ai21labs/Jamba-v0.1)) and 399B in [Jamba 1.5 Large](https://huggingface.co/ai21labs/AI21-Jamba-1.5-Large)),
|
| 8 |
+
and was trained on ~40B tokens.
|
| 9 |
+
|
| 10 |
+
It is great for use in unittests since it is a small model (doesn't take long to download) thas has valid and non-random outputs. Yet, **it did not undergo extensive training and should not be expected to generate high-quality text**.
|