Thanks for your reply. I understood your reply.
But when I look at tmp.requires_grad in the forward() function, it returns True. So you are saying that even though it says True, gradients are not computed. Am I right?