With Encoder-Decoder based models, we can encode the feature from the inputs by encoding and decoding.
My question is how to prevent duplicate gradient calculating.
e.g., input -> enc -> features  ->  dec -> outputs.
Then we would like to obtain the feature of outputs to compute loss like triplet loss: enc(outputs)
Due to enc(outputs) would pass encoder twice, we would like to obtain the gradient for the first time.
I get two options:
1: enc(outputs.detach())
2: torch.no_grad: enc(outputs)
I am not sure which one is the correct way.