Chinchilla’s death
While the article makes good observations, this would appear to be a major oversight by leading research labs if they could have just kept the gas pedal down on simpler models for longer and gotten better performance. This is HackerNews – can we get someone from OpenAI, DeepMind, or MetaAI to respond and justify why cutting off the smaller models at a lower total compute budget is justified?