To clarify, you’re asking that if we get a pretrained network (without freezing any layers) and train it on a transfer task, will the network get worse (or at least, is it possible)?
I would say not necessarily. I would suspect that early on in training, since the last layer was just freshly initialized, that the pretrained network may get “worst”. But I would imagine this would only be early in training. This is just coming from my personal experiences, so take it as you will.