I don’t think there is a strict answer for this because it depends on what the entire model architecture is and not just the backbone. So you can try training with or without freezing though without freezing probably gives you better results.