Llm, Transformers, Fine Tuning, Gpu, Scaling Law.
suggested by OpenAI’s researchers, an interesting way of interpretation of LLMs scaling law is emerged: plot out IsoLoss contours or IsoFLOPs slices, as shown in Fig5. To understand the IsoLoss contours plot on the left, let’s use first principles to grasp the situation. For a given IsoLoss (the