I have 4 networks each being optimized individually. For 3 networks, backprop occurs without any error, but the last one throws the error:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [4096, 3072]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck
The shape of the tensor points me to the last linear layer in one of the network:
class Generator(nn.Module):
    def __init__(self):
        super().__init__()
        self.main = nn.Sequential(
            nn.Linear(256 * 2 + 10 + 1 +512, 4 * 1024),
            nn.BatchNorm1d(2 * 2 * 1024),
            nn.ReLU(True),
            nn.Linear(4 * 1024, 3 * 1024),
            nn.Tanh()
        )
    def forward(self, x):
        b = x.size(0)
        noise = torch.FloatTensor(b, 512).normal_(0, 1).to(x.device)
        noise = make_variable(noise)
        x = torch.cat([x, noise], 1)
        out = self.main(x)
        return out.view(b, 3, 1024, 1)

I don’t see any in place operation here, so what exactly is the issue here? @ptrblck