Sending a tensor to the GPU should not allocate that much system RAM.
Could you post some information about your setup?
I.e. which GPU are you using as well as PyTorch version, how you’ve installed it (built from source or binaries), as well as the local CUDA and cudnn versions, if installed.
I suspect some just-in-time compilation might be going on in the background.
How long does this command take when you first run it?