at the moment I am just using two input images for my training. When I start a training session with the network, pretrained by me, the error increases by some magnitudes (from a few hundred to 10.000 up to 40.000) and commutes than back to the level, where it was at the end of the last session. Through all this the learning rate printed out on the console is always the same, initial one, what makes ne sense to me.
I don’t know what else could be the reason for this big temporal fluctuation of the error.