I am using one trained Module (E) to provide the loss function for the output of another Module (M).
Of course, I would like the errors specifically back-propagated through the specific features of M’s output.
However, I don’t want E to update its parameters at all.
In the init of M, I have tried:
        for param in self.E.parameters():
            param.requires_grad = False

and also
            param.detach()

but after training M a bit, I see that E has been updated. I test this before and after training M by passing some dummy input to E, and seeing that it has changed.
How can I backprop errors from E to M, but keep E fixed?