Which GPU(s) to get for deep learning
I find this article odd with its fixation on computing speed and 8bit.<p>For most current models, you need 40+ GB of RAM to train them. Gradient accumulation doesn't work with batch norms so you really need that memory.<p>That means either dual 3090/4090 or one of the extra expensive A100/H100 options. Their table suggests the 3080 would be a good deal, but it's not. It doesn't have enough RAM for most problems.<p>If you can do 8bit inference, don't use a GPU. CPU will be much cheaper and potentially also lower latency.<p>Also: Almost everyone using GPUs for work will join NVIDIA's Inception program and get rebates... So why look at retail prices?