RedPajama Data
[MPT](https://texonom.com/mpt-bf2abacce4b14fe99929c1fb19ea2c71) 


[RedPajama-Data](https://github.com/togethercomputer/RedPajama-Data)





> [togethercomputer/RedPajama-Data-1T · Datasets at Hugging Face](https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T)

> [RedPajama, a project to create leading open-source models, starts by reproducing LLaMA training dataset of over 1.2 trillion tokens  — TOGETHER](https://www.together.xyz/blog/redpajama)





