Update README.md
Browse files
README.md
CHANGED
|
@@ -15,6 +15,8 @@ GLM-4-32B-Base-32K is an enhanced version of [THUDM's GLM-4-32B-Base-0414](https
|
|
| 15 |
|
| 16 |
This model was developed as a proof-of-concept to validate that a merging-centric approach to context extension can be successfully applied to larger-scale models. The techniques employed resulted in an approximate 5% overall improvement on standard base model benchmarks while significantly improving 32k recall.
|
| 17 |
|
|
|
|
|
|
|
| 18 |
## Model Details
|
| 19 |
- Architecture Base: [THUDM/GLM-4-32B-Base-0414](https://huggingface.co/THUDM/GLM-4-32B-Base-0414)
|
| 20 |
- Parameter Count: 32B
|
|
|
|
| 15 |
|
| 16 |
This model was developed as a proof-of-concept to validate that a merging-centric approach to context extension can be successfully applied to larger-scale models. The techniques employed resulted in an approximate 5% overall improvement on standard base model benchmarks while significantly improving 32k recall.
|
| 17 |
|
| 18 |
+
More details can be found in our blog post [here](https://www.arcee.ai/blog/extending-afm-4-5b-to-64k-context-length) where we applied this work to our upcoming AFM 4.5B
|
| 19 |
+
|
| 20 |
## Model Details
|
| 21 |
- Architecture Base: [THUDM/GLM-4-32B-Base-0414](https://huggingface.co/THUDM/GLM-4-32B-Base-0414)
|
| 22 |
- Parameter Count: 32B
|