I implemented generative adversarial network using both nn.ReLU() and nn.ReLU(inplace=True). It seems that nn.ReLU(inplace=True) saved very small amount of memory.
What’s the purpose of the using inplace=True?
Is the behavior different in backpropagation?