First of all thank you for your fast reply.
Right, For the GT There is no need but my network generate for example 4 point which represent rotated box each point represented by (x,y) then I need to convert the network output to be mask as well to be ale to apply segmentation loss on both the predicted mask (after converting predicted point to mask and the GT mask)
I tried to convert the predicted points to mask using cv2.fillpoly but the loss wasn’t change during training, after investigation I realized that this behaviour because I loss the gredient trace when I convert the predections to numpy to use fillpoly, so I need another way to achieve that to be able to backprop on it
I hope this time I was clear enough