Llm, Transformers, Fine Tuning, Gpu, Scaling Law.
have studied operations research in finance or business school. One can use the blue line to extrapolate what the optimal model size and predicted loss could be, given a larger compute budget, or in other words, more training FLOPs. For example, in the case of the Gopher model, the optimal model