If I don’t crop the image and change
model = maskrcnn_resnet50_fpn(pretrained=True)
to
model = maskrcnn_resnet50_fpn(pretrained=True, max_size=512)
It prevents the model losses to explode and I also don’t get an out of memory error. My model still cannot predict correctly when given the exact same image during inference but maybe I need to experiment more with the parameters.
When I pass the cropped image I don’t face the out of memory issues but the results are not as good as when I pass the uncropped image.