I am training a network using ArcFace loss. My eventual goal is to learn robust embeddings that can differentiate similar and dissimilar images (even for unseen classes).
As this is a verification kind of task, but training ArcFace is a classification task, I am confused about how should I early stop on validation dataset because my validation dataset has unseen classes because validation tasks is of verification. Without early stopping ArcFace loss network overfits on training classification data and gives very poor performance on validation verification task.
Thanks