Some older Tesla products were separate boxes:<p><a href="http:&#x2F;&#x2F;www.nvidia.in&#x2F;object&#x2F;product-tesla-S2050-in.html" rel="nofollow">http:&#x2F;&#x2F;www.nvidia.in&#x2F;object&#x2F;product-tesla-S2050-in.html</a><p>The form factor isn&#x27;t a choice... The cards have a lot of stuff on them that need to go somewhere. For applications that need it, single servers can fit 8 GPUs these days.<p>As for why they&#x27;re still &quot;GPUs&quot;; there is nothing particularly &quot;central&quot; about modern CPUs either. One could argue that neural networks are a form of &quot;graph&quot; though.