Machine Learning, AI, Deep Learning, NLP, Llm.
from 70 million to over 16 billion parameters on 5 to 500 billion tokens. Chinchilla outperforms Gopher and the other large models After finding the relationship between the three factors, they trained a new LLM called Chinchilla which uses same compute budget as 280B Gopher but has 70B parameters