Great question. There are several things going on here. First, a bare CUDA context on a GPU will use about 300MiB VRAM (this gets initialized the first time you call .cuda()). Then libtorch_cuda.so (containing all GPU kernels for various tensor operators, etc.) will be loaded into VRAM taking another 500MiB or so. Finally if you use any cuDNN or cuBLAS functions loading those libraries will also take memory. Additionally, the caching allocator 5 may reserve additional GPU memory ahead of time even before it is required for tensors to save time when more allocations are needed.