there are mainly 2 ways to do this:

Regression bbox and an angle which decribe the rotation of the box;
Directly regression the 4 points as a polygon of the shape. these method have Centernet Cornernet etc can do so.