ptrblck:

If you run your code, check the bias of your last conv layer, as in my runs apparently the activation was “killed” and the model output was a constant value given by the bias.
In my experience this happens if your learning rate is too high and throws the parameters off.


Thank you, that seems to be whats happening here. I tried lowering the learning rate but did not help.
My code is the same as I posted above, just a simple model with sample data. These little issues is what I want to learn and iron out before going to complex models.
Can you please advise how to check the bias and the activations and how would I be able to identify this happening in my models? I tried print(list(model.parameters())) but from the output printed how can I identify that the network stopped learning?
thank you