Thanks !
Now I’m executing the train of the model with the recommendations you made, so now my loss is CosineEmbeddingLoss() and when I use it, actually I had to change it a bit…

Capture.JPG1192×106 20.2 KB

first I had to add a parameter “y”, which I’m not sure what is it for, but apparently its mandatory, so I sent
y = torch.empty(len(sentence)).bernoulli_().mul_(2).sub_(1)
which gives me a tensor of 1 et -1 …
and second I had to remove the mean, as long as I see it as a way to reduce the matrix, the shape of the function seem to be; the two input tensors of same size and the additional parameter “y” one dimension smaller, so my “y” tensor is already dimension 1 (it’s the size of the sentence T) and the two input tensors x1 and x2 are, as you said, size (T , embedding(500)). If I reduce my input vectors with a mean I would have a tensor of dimension 1, with size 500, and the expected “y” needs then to be smaller.
Hope it was clear, I will let you know the results when I have them.