Llm, Transformers, Fine Tuning, Gpu, Scaling Law.
frozen_ops_per_step * frozen_flops It is important to note that this computation only covers the forward pass. For training, the backward pass also needs to be considered, which typically involves a similar amount of computation. If this is getting too complicated, the rule of thumb is compute