alexmarques
commited on
Commit
•
435a6e6
1
Parent(s):
50a00c8
Update README.md
Browse files
README.md
CHANGED
@@ -10,7 +10,8 @@ tags:
|
|
10 |
# SparseLlama-3-8B-pruned_50.2of4
|
11 |
|
12 |
This repo contains model files for a 2:4 (N:M) sparse [Meta-Llama-3-8B](meta-llama/Meta-Llama-3-8B) model pruned in one-shot with [SparseGPT](https://arxiv.org/abs/2301.00774), and then additionally retrained with the [SquareHead](https://arxiv.org/abs/2310.06927) knowledge distillation while maintaining the 2:4 sparsity mask.
|
13 |
-
|
|
|
14 |
|
15 |
## Running the model
|
16 |
|
|
|
10 |
# SparseLlama-3-8B-pruned_50.2of4
|
11 |
|
12 |
This repo contains model files for a 2:4 (N:M) sparse [Meta-Llama-3-8B](meta-llama/Meta-Llama-3-8B) model pruned in one-shot with [SparseGPT](https://arxiv.org/abs/2301.00774), and then additionally retrained with the [SquareHead](https://arxiv.org/abs/2310.06927) knowledge distillation while maintaining the 2:4 sparsity mask.
|
13 |
+
|
14 |
+
**Note:** This is still a work in progress and subject to change. We expect to release new weights with even better accuracy soon.
|
15 |
|
16 |
## Running the model
|
17 |
|