Thanks for the response. As I mentioned, due to large size of the image, parallelizing different layers didn’t solve the issue. I have heavy upsampling layer in the model and memory overshoots there, even if I move computations for the layer to a separate gpu.