Hey,
Thanks for your kind reply, I thought about this problem all day. And you kindly confirmed to me that they are not code errors!
My network uses 3D kernels so I probably have many parameters to optimise, a fundamental role is played by the lr. Analysing the behaviour of the gradient after transfer learning on two different parts of the model using a SGD, with momentum = 0.9, weight_decay = 1e-3, nesterov = True.
I have the following behaviors to vary the learning rate.
lr = 1e-3 the model does not optimize
lr = 1e-2 begins to rise with accuracy.
Now with 1e-2 after transfer learning I have the following values ​​(where the maximum is 100)
epochs
0.0
18.02626847989203
19.78656745645463
14.658616073090036
17.290238624269023
18.621533335401455
17.22693140700638
27.07915645534009
13.420461302124785
As you can see, the network at the second epoch already starts at 18% accuracy, but with a lr = 1e-3 I have the following behavior:
0.0
0.0
0.0
0.0
0.0
Which turns out to be comparable to the network without transfers.
So the question is spontaneous to me. Is there a correlation between the speed of  parameters updating  and the re-adapting of the network?