I tried to detach, lets say this is my network:

out4 = x1 + self.rcu4(x1)#fourthstep
out4 += self.rcu4(out4)
out4 += out3
out4 = self.respool4(out4)
out4 += self.rcu4(out4)
out4 = self.multires_end(out4)
out4.detach()
    out4 = self.m(out4)
    return out4


I still have an error :

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation