What kind of medical images are you dealing with?
Since you are dealing with flow I would assume some kind of US doppler?
Depending on your image modality you might find some open source data sources.
If your use case is academic it might be even simpler, as not so many are on CC0.
How many videos / images do you have?
Maybe you could indeed start with a standard pre-trained model and fine tune through all layers, if you have enough images.