Hi @ptrblck, I have some doubts about the code posted by the person who created this thread.
I could understand till  this line:
# for numerical stability

Can you please explain the remaining code?
That loss function (class SupConLoss(nn.Module):) is based on the equation no.4 in this paper.
upon comparing the equation and the code, I don’t know where exactly in the code they are separating`` Zi, Zj and Zk values for each Li value and moreover I don’t see any loop which would iterate over all values of i. So, I guess that computes loss only for the first image of the batch.
In this file, I don’t know what line no.231 (losses.update(loss.item(), bsz) does. I guess it’s just for printing purpose only.
Thank you for sparing your time.