Thanks a lot for your reply.
I don’t use nvidia-smi. Maybe it’s more of a Linux thing? Anyway, I do monitor memory usage with GPU-Z which shows a plot over time. In GPU-Z I can clearly see the GTX 1060 6 GB reaching almost its full memory capacity (5.9 GB). The RTX 2070 8 GB never approaches its 8 GB capacity.
I’d say that the 2 MB allocation cannot be the culprit. The biggest question is how come there are only 1.85 MB free when in theory there should be 8 GB - 6.32 GB = 1.68 GB free? I understand memory fragmentation may be an issue sometimes, but it’s probably not the case here since I can fill up the 1060’s memory just fine.
What do you think of the code allocating tensors on the GPU? Playing around with the tensor sizes and monitoring memory usage with GPU-Z I can push the 1060 to use 5.9 / 6 GB but I can only get the 2070 to 6.9 / 8 GB. Why is that? Could the memory chips be faulty? This is my biggest problem and a huge disappointment for a brand new card. As I mentioned in the previous post, I tried switching the cards, removing the 1060, fresh install of drivers, conda environment, different versions of PyTorch, and nothing worked.