Hi @ptrblck,
I am working on the same loss function.
But my input to loss function (Zi, Zj, Zk from equation no.4 in this paper ) are feature maps which has size of (num_features, h, w) = (128,8,8).
I looked at above code of loss function implementation:
class SupConLoss(nn.Module):
But I have no idea what’s going on in that function and moreover their input size is different. So, I was thinking to code the mathematics of that equation which is very complex. I will need several for loops and some logic to differentiate i, j and k.
Batch size=100, so for each i (0 to 99), I have 9 different j values (because in 1 batch there are 9 images having same label as 'i’th image) making k=90(90 images having labels different than that of ‘i’th image). Like wise I have to go through all the images’ feature maps. Can you please suggest how should I proceed?
In addition to that I have doubt on, how to compute dot product of Zi having size of [128,8,8] and Zj of same size which should give me a single value (a scalar value)?