So when I ran the code you have pasted @ptrblck , the outputs were different, are you saying that there might be some issue?
Output:
tensor([[ 0.5963,  0.1013, -1.2797, -0.4520, -0.7417, -0.5781, -0.0526,  0.9478,
         -1.3636, -0.2064]], grad_fn=<AddmmBackward>)
tensor([[ 0.4647, -0.3197, -0.1019,  0.2178, -0.7557,  0.3196,  0.7327,  0.0902,
         -0.2639,  0.0036]], grad_fn=<AddmmBackward>)

But there seems to be some issue here, as the self.relu has not been applied for the first output (lets call that A  and second one which has an inplace set to true as B ) why is that I can’t get back B if I do relu on A i.e,

B = self.relu(A)

Am I missing anything or is there any bug in the codebase?