Hi Opacus community,
I am looking for experiences / best practices for using DP with transfer learning. Let’s say a hospital decides to build a DP image classification model based on patient data.
3 scenarios com to mind:

baseline network pre-trained on public dataset (from a similar domain, e.g. x-rays)
baseline network pre-trained on existing data of the hospital (classic, non private way)
baseline network pre-trained with DP on existing data of the hospital

I would assume that generally all approaches could make sense because probably less epochs are required compared to training from scratch (hence, spending less privacy budget).
I would appreciate learning your experiences / thoughts about these scenarios.
Thanks
Andreas