Hi, all the samples that I am using have are valid. The issue that I am facing is when the x1 (xmax) value is smaller than x2 (xmin), the loss rpn_box_reg loss becomes NaN. For example, for the image below, the bouding boxes are tensor([[ 53., 89., 7., 226.]]) i.e. [x2, y2, x1, y1]. When the x1 value is smaller than x2, the loss goes to zero, however, it works fine when x1>x2. In fact, it trains quite well. As you can see, the values are correct as the cyclist has the correct bounding box based on the values above. I hope this makes it a bit more clear as to the issue I am facing.

test.jpg720×720 103 KB