@KFrank Thanks for your suggestions. Following your recommendations, I narrowed down the part causing this issue. Turns out in a layer block I used torch.nn.ReLU(True) without paying much attention to it. The issue occurred since I explicitly performed this inplace ReLU operation.