Llm, Transformers, Fine Tuning, Gpu, Scaling Law.
are more sample-efficient than small models, reaching the same level of performance with fewer optimization steps and using fewer data points. It is evident that, for a given tokens processed, larger models achieve lower test loss, as shown in Fig3. Fig4: DeepMind suggests a modified scaling law,