Setting np.random.seed(123) should take care of always producing the same sample

Unless later in the network, you generate more random numbers. Then depending if you run these layers, you will get different random inputs.

TODO: Explain why it makes a difference when the following

Maybe the layer just after the conv is an activation function that is applied inplace. And so it changes the output of the conv. You can clone the result to avoid this with: self._conv_output = x[0, self.selected_filter].clone().