AI Is Catapulting Nvidia Toward the $1 Trillion Club
Am I the only one thinking that NVIDIA doesn&#x27;t <i>really</i> have a moat here?<p>How many A100 or H100 cards are actually manufactured annually? A few hundred thousand, if that?<p>Suddenly, there&#x27;s a big demand. Microsoft mentioned buying something like 25,000 of the H100 cards for GPT-4 and ongoing training. I&#x27;m certain they&#x27;re not paying retail pricing, so that&#x27;s a few hundred million in revenue for NVIDIA. They&#x27;re probably the biggest single customer right now, except perhaps for Amazon.<p>NVIDIA&#x27;s revenue in 2022 was $27 billion. The <i>additional</i> H100 cards they&#x27;ve sold this year is a fraction of that. Their retail prices have spiked and availability has dropped because supply is inelastic and there aren&#x27;t any other suppliers with equivalent products... <i>yet</i>.<p>Fundamentally, a H100 is not <i>that</i> different to a desktop GPU! It&#x27;s a little bigger, the math units have a different ALU balance, and they use high-bandwidth memory (HBM), but <i>that&#x27;s it</i>. There&#x27;s nothing else really special about them. Unlike a CPU, which is <i>extremely complex</i>, a GPU is a relatively simple unit repeated over 10K times. In some sense, it&#x27;s a copy-paste exercise.<p>NVIDIA has a tiny moat, because AMD simply <i>didn&#x27;t bother</i> to go after what was -- until now -- a relatively small market.<p>That market is going to be <i>huge</i>, but that invites competition! When tens or even hundreds of billions are on the table, you can bet your bottom dollar that AMD, Intel, Google, and even Facebook won&#x27;t sit idly by and watch NVIDIA walk off with it.<p>So what moat does NVIDIA have?<p>CUDA is like assembly language. PyTorch can target other back-ends. Compilers can target other GPU instructions sets. Throw a <i>billion dollars</i> at this, and it suddenly becomes an eminently solvable problem. Just look at Apple&#x27;s CPU transitions and Amazon rolling out ARM cloud servers.<p>A card with HBM memory? AMD did that first! They already have the tech.<p>A GPU + CPU hybrid with unified memory? Both Intel and Apple have either existing or upcoming products. Intel for example just abandoned[1] a HPC CPU design that was a combo of a GPU+CPU surrounded by HBM chips acting as a cache for terabytes of DDR5 memory -- ideal for training or running very large language models!<p>A GPU with a huge amount of 16-bit, 8-bit, or 4-bit ops&#x2F;sec? Guess what: this is <i>easier</i> than getting high performance with 64-bit floats! You can literally <i>brute force</i> optimal circuit layouts for 4-bit ALUs. No need to be clever at all. All you need is the ability to manufacture &quot;3nm&quot; chips. TSMC does that, not NVIDIA. Intel and Samsung are catching up, rapidly.<p>Fundamentally, the programming interface 99% of AI researchers use are high-level languages like Python or maybe C++. Compilers exist. Even with CUDA, diverse instruction sets and capabilities exist.<p>So.. where&#x27;s the moat!?<p>[1] Ooo, I bet they feel <i>real stupid</i> right now for throwing in the towel literally months before the LLM boom started taking off.