it comes down to two cuda kernels that, strongly simplified, look like:
mul(float* a, float* b)
vs
mul(float* a, float b)
second one is marginally faster, as one “load from global memory pointer” operation is avoided
but this is not related to CPU-GPU transfers, note that function arguments, launch configuration and a signal to execute cuda kernel itself require a data transfer anyway (I’d assume it is a single transfer that uses faster “constant memory”, but haven’t investigated into this).
takeaway is, it is better to NOT use tensors, when tensor-scalar operator versions exist (they usually do when you don’t need to wrap numbers in tensors). OTOH, the timing difference is negligible.