Nvidia H100 GPUs: Supply and Demand
The real gut-punch for this is a reminder how far behind most engineers are in this race. With web 1.0 and web 2.0 at least you could rent a cheap VPS for $10/month and try out some stuff. There is almost no universe where a couple of guys in their garage are getting access to 1000+ H100s with a capital cost in the multiple millions. Even renting at that scale is $4k/hour. That is going to add up quickly.<p>I hope we find a path to at least fine-tuning medium sized models for prices that aren't outrageous. Even the tiny corp's tinybox [1] is $15k and I don't know how much actual work one could get done on it.<p>If the majority of startups are just "wrappers around OpenAI (et al.)" the reason is pretty obvious.<p>1. <a href="https://tinygrad.org/" rel="nofollow noreferrer">https://tinygrad.org/</a>