Anyone know how to calculate the area of intersection between two (convex) polygons in a differentiable manner so it can be used as a loss function for backpropogation? This is a fairly obvious extension of rectangular bounding box intersection-over-union as a loss function but I haven’t seen anyone do it yet. You could imagine scenarios in which you are attempting to identify a rectangular face of an object within an image; if the object is at an angle the rectangle will be a skewed and become some other quadrilateral. Just thought I’d ask before I attempt it myself.