Thank you for your quick reply. I have tried reducing the learning all the way down to 0.00001, but i continue to get the same issue. This is the settings:
params = [p for p in model_ft.parameters() if p.requires_grad]

optimizer = torch.optim.SGD(params, lr=0.00001,
                            momentum=0.9, weight_decay=0.0005)

lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer,
                                               step_size=1,
                                               gamma=0.1)

I have noticed that this only seems to emerge when xmax value is lower than xmin value. Would you have any idea why this may be the case?
I will have a look at the rpn_box_regression as you suggested to see if the values are high.