Update README.md
Browse files
README.md
CHANGED
@@ -13,7 +13,7 @@ base_model:
|
|
13 |
## Model summary
|
14 |
|
15 |
This model is part of the π [FineMath](https://huggingface.co/datasets/HuggingFaceTB/finemath) ablations, we continue pretraining [Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B) base on different math datasets for 60B tokens.
|
16 |
-
The model has 3.21B parameters and 4096 context length. It was trained on **
|
17 |
|
18 |
- **License**: Apache-2
|
19 |
- **Languages**: English
|
|
|
13 |
## Model summary
|
14 |
|
15 |
This model is part of the π [FineMath](https://huggingface.co/datasets/HuggingFaceTB/finemath) ablations, we continue pretraining [Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B) base on different math datasets for 60B tokens.
|
16 |
+
The model has 3.21B parameters and 4096 context length. It was trained on **60B tokens** using a mix of 50% FineMath-3+ and 50% InfiWebMath-3+ from the π [FineMath](https://huggingface.co/datasets/HuggingFaceTB/finemath) dataset.
|
17 |
|
18 |
- **License**: Apache-2
|
19 |
- **Languages**: English
|