harryhan618:

nn.ReLI(inplace=True)


That’s a good question! I think that this was an initial limitation based on the PyTorch (whitepaper) manuscript on arxiv. But based on what I’ve seen (e.g., by @soumith https://gist.github.com/soumith/71995cecc5b99cda38106ad64503cee3 216) it seems that inplace ops like nn.ReLU(inplace=True) are supported in the autodiff engine now. Not sure, but I guess the same should be true for the functional one as it is referencing the nn. one.