pierremrg:

c = len(a) # tensor c will still be on GPU


c is not a Tensor here but a python number. And as such it will be on the CPU.
For you other questions:


.float() makes a full copy if the given Tensor is not of floating type, otherwise it returns the input as-is
All operations on GPU Tensors will happen on GPU. + or add() are the same thing so you can use either.

len() returns a python number. So the result will always be on CPU (as python number cannot be on GPU). And it returns the size of the first dimension, which doesn’t need access to the content of the Tensor, only its metadata, which are read on the CPU side.
The two lines are very similar. The first one is preferable as a single plain number can be sent as an argument to the GPU kernel which will be faster than creating a full Tensor containing a 1. and send that to the GPU.

Some other pointers for perf:

PyTorch will never copy the content of a Tensor between device for you. Only when you call .cuda() or .cpu(), etc where a copy will happen.
It is better to create the Tensor you want on the right device directly to avoid extra copies: torch.tensor([1, 2, 3]).float().cuda() should be torch.tensor([1, 2, 3], dtype=torch.float, device="cuda")