In the official notebook, the last fully connected layer has an sigmoid activation function. self.fc2 doesn’t have any. And because the output isn’t mapped between 0 and 1 so maybe your gradients are different.