First, my special thanks go to you.
You are right that’s the NLP task with a siamese network to find the similarity between two sentences. I compute loss by code below. Besides, How can I deal with overfitting in my case well? I found out that augmentation can help, but any suggestions are welcomed.
The reason for jump is that, after a few epochs, if loss gets worse, then I restore the best previous model and decrease the learning rate by multiplying 0.85.  (AdamW(MM.parameters(), lr=0.001))
As mentioned earlier, 4500 train samples and 500 eval samples of a dataset for the first plot are used. For the second I altered train samples with new 750 ones from another dataset
LOSS is computed like this:
for i, (xA, xB, score) in enumerate(data_loader, 1):
   total_loss += (loss.item()*len(xA))

epoch_loss = total_loss / len(data_loader.dataset)

Is it OK for showing the result, or should I use cumulative loss like (loss.item() / i ) * len(A) ??