i made a little modification in the code. I change the relu function as a single operation x = self.relu(x), and then return x. In this way i can observe the input change accurately. But it behaves like above.