tlim:

From testing experience, the first Tensor push to GPU will roughly take up to 700-800 MiB of the GPU VRAM. You can then allocate more tensor to GPU without a shift in the VRAM until you have exceeded the pre-allocated space given from the first Tensor.


No, that’s not the case. PyTorch will create the CUDA context in the very first CUDA operation, which can use ~600-1000MB of GPU memory depending on the CUDA version as well as the used device.
PyTorch itself will allocate the needed memory and will use an internal cache mechanism. You can read more about it here 7.



 tlim:

How do I create the first Tensor without 700+MiB worth of VRAM baggage?


That’s not possible, since the CUDA context has to be created, which contains e.g. the device code.