The Mathematics of Training LLMs
this was the deepest dive we ever did into Transformers Math 101 (<a href="https://news.ycombinator.com/item?id=35631546">https://news.ycombinator.com/item?id=35631546</a>). we mostly cover AI Engineer/inference time things on the pod, but training knowledge is hard won and extremely valuable, and I love it when experts are wiling to distill the rules of thumb for what they have learned.<p>questions welcome!