Solved by albanD in post #2 
              
              
                The backend use a custom cuda allocator. So allocating cuda tensors is very efficient if you do the same allocations repeatedly. So it’s ok for your forward to create new tensors.