Is that based on FasterRCNN? I’m trying to replicate on Pascal VOC2007. I did test against a simple Pytorch version which converges very fast as expected. I don’t see a major difference in my code except that my ground truth values are stored in a large map and so my loss functions have to perform a bit of tensor slicing and reshaping to get at the y_true values.