alex83803:

If I understand well, I have to spilt my input data (image) In two/four, send the splitter data to the different gpus and merge the data after process completion is that correct?


No, this would be the the data parallel approach.
For model sharding, you could keep only parts of the model on each device and would transfer the forward activation to the corresponding devices.



 alex83803:

Could you show me how you would use Model parallel in this precise case? My input image is a jpg image.


My example code snippet shows how the model should be manipulated for a (manual) model sharding.
However, I think higher level APIs such as Ignite, Lightning, Catalyst etc. would already have an (automatic) utility for it, so you could also see how they’ve implemented it.