My network applies modules a, b and c upon the input in that order.
c looks like:
    def forward(self, anch, pos, neg):
        pos_out = self.fc(anch) + self.fc(pos)
        pos_out = to.sigmoid(self.fc_final(fu.relu(pos_out)))
        neg_out = self.fc(anch) + self.fc(neg)
        neg_out = to.sigmoid(self.fc_final(fu.relu(neg_out)))
        binary_loss = (-to.log(pos_out + 1e-7) - to.log(1 - neg_out)).mean()
        return binary_loss

When I wrap b (just does some indexing operations) in torch.no_grad(), I get an error RuntimeError: No grad accumulator for a saved leaf!. It works when I don’t wrap it, or when I change forward to concat the inputs before calling self.fc (a fully connected layer, of course the input dimension is changed when I do that).
What’s going on? And is wrapping with torch.no_grad() the right thing when I don’t want some operations to have an effect on the gradient?