Using a pretrained model shouldn’t hurt your training, as you could see it as a “good initialization” of the parameters in comparison to a random initialization.
From my experience, the model loss might blow up and you would need to retrain the model, but this shouldn’t be worse than training from scratch.
An advantage and also a shortcoming from using a pretrained model is the fixed architecture.
While it should be no problem to change e.g. the final classifier, I’m not sure which approach would work best e.g. for transforming a 2D model into a 3D one.