that’s interesting, can you explain why is that crucial?
why must i do optimizer = Adam(model.parameters after loading the pretrained weights)
anyway, this is not the problem in my case, since i also don’t see training when trying to train the network without loading weights.
i saw some post about a similar issue that was solved by changing a bunch of leaky relus to relus. i suspect this is a problem i’m facing.
the original net works with leakyRelu and then i have a new net composed of the first half of the first net followed by a flatten layer, a linear layer and a Relu. the output of this relu is one neuron and the loss is MSE.
it’s always returning 0. so that’s why i think there is no learning.
but there is also bias so why doesn’t the net simply learn to add bias and to get a positive outcome?