Machine Learning, AI, Deep Learning, NLP, Llm.
10× smaller. These models are not the fastest to train but they are faster at inference. Pre-training data They have used byte-pair encoding. For 6B and 13B parameter models, they have trained with 1T tokens. For 32B and 65B parameter models, they have trained with 1.4T tokens. They have used the