Suppose I have myNetwork class defined as:
class myNetwork(nn.Module):
    def __init__(self, a, b, embedding_dim):
        super(modeler, self).__init__()
        self.embed1 = nn.Embedding(a, embedding_dim)
        self.embed2 = nn.Embedding(b, embedding_dim)

        
    def forward(self, idx1, idx2):
        
        embeds1 = self.embed1(idx1)
        embeds2 = self.embed2(idx2)

        tmp = embeds1 * embeds2 

        tmp1 = embeds1 + tmp

        output = torch.sum((tmp1 - embeds2)**2,1)

        return output

In my training loop, I forward idx1 and idx2 to get output, which is a Variable.
As shown, tmp and tmp1 are intermediate Variables whose gradients are not necessary, because I can write the following instead by substituting tmp and tmp1 as follows:
output = torch.sum(((embeds1 + (embeds1 * embeds2)) - embeds2)**2,1)

My question is, since tmp and tmp1 do not require gradients, is it more efficient to detach tmp and tmp1 in forward() as below?
....
tmp = tmp.detach()
tmp1 = tmp1.detach()
....

I am new to pytorch and hope I don’t bother you.
Thanks