For example, if I need to implement a matchnet, which requires two samples to pass through two models with shared weights, and compute the loss with the two outputs. If I need to try this with pytorch, what is the correct way to do this ?
I am thinking that maybe I could run forward path two times with the two input samples and then compute the loss and run backward path. But I do not if this is the correct option, since, for some modules, the backward options need the forward path input, and if I run forward two times with two different inputs, the backward process is likely to be disturbed. So what is the good way to do this ?