How did you define your target containing the corners of your bounding box?
Did you define it as a “segmentation map” containing 1s at the corner pixels and 0 elsewhere?
If so, you could try to use the position of the prediction to determine which pixel is “top left” and which is “bottom right”. However, since your prediction can output multiple pixels, you would have to deal somehow with this ambiguous result (maybe filtering out invalid candidates?).
I think the usual workflow to predict a bounding box would be to output the coordinates directly instead of a heat map.