Your standalone model works and I’m unable to reproduce the issue in the forward pass:
model = PatchDiscriminator()
x = torch.randn(1, 3, 96, 96, requires_grad=True)
out = model(x)
x = torch.randn(1, 3, 96, 96, requires_grad=True)
out = model(x.detach())