Safetensors
English
llama
loubnabnl HF staff commited on
Commit
c0feb87
β€’
1 Parent(s): 13c1c42

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -13,7 +13,7 @@ base_model:
13
  ## Model summary
14
 
15
  This model is part of the πŸ“ [FineMath](https://huggingface.co/datasets/HuggingFaceTB/finemath) ablations, we continue pretraining [Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B) base on different math datasets for 60B tokens.
16
- The model has 3.21B parameters and 4096 context length. It was trained on **160B tokens** using a mix of 40% [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) and 30% FineMath-3+ and 30% InfiWebMath-3+ from the πŸ“ [FineMath](https://huggingface.co/datasets/HuggingFaceTB/finemath) dataset.
17
 
18
  - **License**: Apache-2
19
  - **Languages**: English
 
13
  ## Model summary
14
 
15
  This model is part of the πŸ“ [FineMath](https://huggingface.co/datasets/HuggingFaceTB/finemath) ablations, we continue pretraining [Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B) base on different math datasets for 60B tokens.
16
+ The model has 3.21B parameters and 4096 context length. It was trained on **60B tokens** using a mix of 50% FineMath-3+ and 50% InfiWebMath-3+ from the πŸ“ [FineMath](https://huggingface.co/datasets/HuggingFaceTB/finemath) dataset.
17
 
18
  - **License**: Apache-2
19
  - **Languages**: English