@Konpat_Ta_Preechakul Thanks for the reply.
I went and trained the model for about 2500 epochs. Here are the losses of the last 10 epochs:

[tensor([ 3.6269], device=‘cuda:0’),
tensor([ 3.6266], device=‘cuda:0’),
tensor([ 3.6263], device=‘cuda:0’),
tensor([ 3.6259], device=‘cuda:0’),
tensor([ 3.6257], device=‘cuda:0’),
tensor([ 3.6254], device=‘cuda:0’),
tensor([ 3.6249], device=‘cuda:0’),
tensor([ 3.6247], device=‘cuda:0’),
tensor([ 3.6243], device=‘cuda:0’),
tensor([ 3.6240], device=‘cuda:0’)]

The loss has significantly gone down. How much loss is acceptable? It may depend on the end application but I was wondering if I could get a yardstick.
Here is the head of the vectors:
==> before.vec <==
97 10
shall -1.51200 -0.13722 -0.02999 -0.29068 -0.43199 -0.41440 -1.26230 -0.00776 -0.90202 -0.47192
sum 0.04348 -0.17251 -0.32429 0.59110 0.58860 -0.63346 0.93111 -1.62142 -2.48102 0.89572
all -1.96656 1.74516 0.28631 0.21345 0.67012 1.27416 0.11448 -1.30043 -2.47435 0.11154
When -0.29002 0.21662 -0.99376 -1.33235 -0.72533 1.38633 -0.24751 -0.12052 -1.33563 -0.36146
art 0.77356 -0.97083 1.50570 -0.32661 -0.12820 -0.24594 2.36714 -0.47082 1.05813 -1.52117
succession 0.47500 0.70047 -0.47240 -0.39529 -0.22532 -0.11470 -0.43629 0.97439 1.50639 -0.66178
held: 1.23583 0.42217 0.53700 -0.72943 0.56429 -1.65103 -0.68944 1.54767 0.32436 1.65500
much 0.96233 -1.03391 -1.68684 -0.38755 0.16864 -0.49799 -0.34811 -2.36278 0.82771 1.00832
'This -0.09349 1.23545 0.62555 0.20450 -1.33267 0.14939 -1.50295 -1.37122 -1.14798 1.31911

==> after.vec <==
97 10
shall -1.80875 -0.27009 0.04512 -0.35795 -0.40817 -0.52694 -1.32084 0.03620 -1.05129 -0.51756
sum 0.03460 -0.15234 -0.32818 0.64060 0.65705 -0.63437 1.10220 -1.75300 -2.66330 0.95863
all -2.16454 1.86197 0.36141 0.23959 0.70754 1.38740 0.11906 -1.38467 -2.69791 0.10806
When -0.28593 0.23379 -1.06475 -1.41304 -0.75584 1.46333 -0.28699 -0.10579 -1.39061 -0.34203
art 0.77069 -1.02793 1.58267 -0.30294 -0.12727 -0.23977 2.47910 -0.50910 1.12315 -1.60805
succession 0.57382 0.80211 -0.56150 -0.42444 -0.24887 -0.07652 -0.52180 1.00314 1.64484 -0.75496
held: 1.32437 0.43335 0.53349 -0.77039 0.58206 -1.80131 -0.75251 1.65968 0.32944 1.74611
much 1.02373 -1.11061 -1.75183 -0.41527 0.20750 -0.46269 -0.32175 -2.53160 0.87068 1.03427
'This -0.09142 1.29842 0.65856 0.15368 -1.41159 0.14917 -1.56514 -1.44337 -1.23931 1.35460

There are much more differences here than the previous post. However, the variations are small. Is this normal? In my head, I’m thinking that the starting embeddings and the ending embeddings would be drastically different but I can only see very small changes.