Llm, Transformers, Fine Tuning, Gpu, Scaling Law.
achieve the same model performance with less model size, as shown in the left of Fig4. Fig5: Parametric fit loss function Here comes my favorite part: When the researchers at DeepMind modeled the loss as a function of model size and number of tokens, and used the constraint FLOPs (N, D) ~ 6ND,