Solved by Tejan_Mehndiratta in post #3 
              
              
                Yes, It’s True. If you transfer a tensor on Cuda and then do any operation on that tensor, the operation happens on Cuda. 
Thanks.