Hi,
I am trying to implement a Dependency Parsing model using the transformer model in here with a few changes.
On the training, my loss has decreasing trend; but the predictions at the end of 20 epochs are not meaningful.
What could be the problems?
Thanks for any help in advance.
Outputs are below here:
The training:

Epoch 0
Epoch Step: 1 Loss: 12.520 Tokens per Sec: 156.228836
Epoch Step: 51 Loss: 9.549 Tokens per Sec: 144.077591
Epoch Step: 1 Loss: 2.845 Tokens per Sec: 139.670441
Epoch Step: 51 Loss: 7.709 Tokens per Sec: 148.557648
tensor(0.0452)
Epoch 1
Epoch Step: 1 Loss: 18.100 Tokens per Sec: 132.820221
Epoch Step: 51 Loss: 10.469 Tokens per Sec: 151.330307
Epoch Step: 1 Loss: 8.830 Tokens per Sec: 141.060349
Epoch Step: 51 Loss: 9.055 Tokens per Sec: 146.867462
tensor(0.0348)
Epoch 2
Epoch Step: 1 Loss: 11.577 Tokens per Sec: 143.026764
Epoch Step: 51 Loss: 4.532 Tokens per Sec: 133.869629
Epoch Step: 1 Loss: 1.540 Tokens per Sec: 166.832520
Epoch Step: 51 Loss: 0.617 Tokens per Sec: 133.209915
tensor(0.0191)
Epoch 3
Epoch Step: 1 Loss: 5.193 Tokens per Sec: 101.057816
Epoch Step: 51 Loss: 3.458 Tokens per Sec: 125.874458
Epoch Step: 1 Loss: 4.809 Tokens per Sec: 146.683258
Epoch Step: 51 Loss: 8.953 Tokens per Sec: 132.177780
tensor(0.0413)
Epoch 4
Epoch Step: 1 Loss: 11.321 Tokens per Sec: 115.404495
Epoch Step: 51 Loss: 3.964 Tokens per Sec: 129.871109
Epoch Step: 1 Loss: 5.374 Tokens per Sec: 153.859207
Epoch Step: 51 Loss: 5.484 Tokens per Sec: 119.325813
tensor(0.0279)
Epoch 5
Epoch Step: 1 Loss: 7.173 Tokens per Sec: 115.805588
Epoch Step: 51 Loss: 1.819 Tokens per Sec: 123.073883
Epoch Step: 1 Loss: 0.960 Tokens per Sec: 170.719009
Epoch Step: 51 Loss: 1.785 Tokens per Sec: 125.758141
tensor(0.0167)
Epoch 6
Epoch Step: 1 Loss: 0.949 Tokens per Sec: 143.089218
Epoch Step: 51 Loss: 5.166 Tokens per Sec: 119.562157
Epoch Step: 1 Loss: 1.305 Tokens per Sec: 145.756500
Epoch Step: 51 Loss: 3.682 Tokens per Sec: 124.026428
tensor(0.0146)
Epoch 7
Epoch Step: 1 Loss: 3.911 Tokens per Sec: 138.809906
Epoch Step: 51 Loss: 5.827 Tokens per Sec: 122.372780
Epoch Step: 1 Loss: 0.494 Tokens per Sec: 137.776642
Epoch Step: 51 Loss: 2.666 Tokens per Sec: 131.723038
tensor(0.0121)
Epoch 8
Epoch Step: 1 Loss: 2.576 Tokens per Sec: 136.597214
Epoch Step: 51 Loss: 2.826 Tokens per Sec: 126.036842
Epoch Step: 1 Loss: 2.027 Tokens per Sec: 126.010918
Epoch Step: 51 Loss: 3.742 Tokens per Sec: 130.350937
tensor(0.0090)
Epoch 9
Epoch Step: 1 Loss: 1.823 Tokens per Sec: 142.635910
Epoch Step: 51 Loss: 2.043 Tokens per Sec: 125.544189
Epoch Step: 1 Loss: 0.558 Tokens per Sec: 133.733505
Epoch Step: 51 Loss: 1.802 Tokens per Sec: 129.989548
tensor(0.0087)
Epoch 10
Epoch Step: 1 Loss: 1.817 Tokens per Sec: 128.555618
Epoch Step: 51 Loss: 5.115 Tokens per Sec: 121.563423
Epoch Step: 1 Loss: 0.555 Tokens per Sec: 145.560699
Epoch Step: 51 Loss: 2.464 Tokens per Sec: 135.761688
tensor(0.0088)
Epoch 11
Epoch Step: 1 Loss: 0.869 Tokens per Sec: 159.163788
Epoch Step: 51 Loss: 2.623 Tokens per Sec: 124.412704
Epoch Step: 1 Loss: 0.772 Tokens per Sec: 148.348251
Epoch Step: 51 Loss: 0.438 Tokens per Sec: 127.859993
tensor(0.0078)
Epoch 12
Epoch Step: 1 Loss: 2.377 Tokens per Sec: 123.196060
Epoch Step: 51 Loss: 1.719 Tokens per Sec: 122.183594
Epoch Step: 1 Loss: 1.415 Tokens per Sec: 124.849358
Epoch Step: 51 Loss: 1.805 Tokens per Sec: 132.770676
tensor(0.0080)
Epoch 13
Epoch Step: 1 Loss: 1.472 Tokens per Sec: 124.035645
Epoch Step: 51 Loss: 0.777 Tokens per Sec: 127.847717
Epoch Step: 1 Loss: 0.605 Tokens per Sec: 140.335785
Epoch Step: 51 Loss: 0.380 Tokens per Sec: 133.239578
tensor(0.0061)
Epoch 14
Epoch Step: 1 Loss: 1.652 Tokens per Sec: 116.933731
Epoch Step: 51 Loss: 2.005 Tokens per Sec: 131.188080
Epoch Step: 1 Loss: 0.057 Tokens per Sec: 162.092529
Epoch Step: 51 Loss: 0.840 Tokens per Sec: 131.577316
tensor(0.0066)
Epoch 15
Epoch Step: 1 Loss: 0.573 Tokens per Sec: 144.241257
Epoch Step: 51 Loss: 1.395 Tokens per Sec: 133.703445
Epoch Step: 1 Loss: 0.922 Tokens per Sec: 150.064331
Epoch Step: 51 Loss: 1.862 Tokens per Sec: 133.412491
tensor(0.0062)
Epoch 16
Epoch Step: 1 Loss: 1.002 Tokens per Sec: 141.052292
Epoch Step: 51 Loss: 0.367 Tokens per Sec: 127.523247
Epoch Step: 1 Loss: 0.792 Tokens per Sec: 137.230988
Epoch Step: 51 Loss: 1.554 Tokens per Sec: 132.268616
tensor(0.0057)
Epoch 17
Epoch Step: 1 Loss: 1.520 Tokens per Sec: 139.651688
Epoch Step: 51 Loss: 0.979 Tokens per Sec: 121.396866
Epoch Step: 1 Loss: 0.826 Tokens per Sec: 125.688881
Epoch Step: 51 Loss: 1.198 Tokens per Sec: 138.106827
tensor(0.0057)
Epoch 18
Epoch Step: 1 Loss: 0.738 Tokens per Sec: 142.477478
Epoch Step: 51 Loss: 0.467 Tokens per Sec: 128.806183
Epoch Step: 1 Loss: 0.153 Tokens per Sec: 155.590027
Epoch Step: 51 Loss: 1.191 Tokens per Sec: 136.467148
tensor(0.0050)
Epoch 19
Epoch Step: 1 Loss: 0.456 Tokens per Sec: 133.751083
Epoch Step: 51 Loss: 0.315 Tokens per Sec: 128.097702
Epoch Step: 1 Loss: 0.584 Tokens per Sec: 151.586060
Epoch Step: 51 Loss: 0.524 Tokens per Sec: 129.905441
tensor(0.0051)

The predictions:

the reality is that it stands for and operationalizes us power , in cooperation with america 's closest allies .
Eisner Prediction:  root 's 's 's 's 's 's 's 's 's 's 's 's 's 's 's . 's 's the
Prediction:	1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609
Target:	reality is root stands stands is stands operationalizes stands power operationalizes stands cooperation stands allies allies america allies cooperation is



there are parallel shortcomings in many other markets .
Eisner Prediction:  root many many many many . many many there
Prediction:	1609 1609 1609 1609 1609 1609 1609 1609 1609
Target:	are root shortcomings are markets markets markets are are



the victorians produced his plays as lavish spectacles on a grand scale .
Eisner Prediction:  root plays plays plays the . . . . . . . the
Prediction:	1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609
Target:	victorians produced root plays produced spectacles spectacles produced scale scale scale produced produced



what is the minimum password strength ?
Eisner Prediction:  root what strength strength strength what strength
Prediction:	1609 1609 1609 1609 1609 1609 1609
Target:	root what strength strength strength what what



license .
Eisner Prediction:  root license
Prediction:	1609 1609
Target:	root license



if you already have a facebook account , you can log in to your account from the same page .
Eisner Prediction:  root the the the the the the the the the the the the the the the page the if if
Prediction:	1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609
Target:	have have have log account account have have log log root log account account log page page page log log



there is no oppression of russian speakers in ukraine , and there never has been .
Eisner Prediction:  root . . . . . . . . . . . . . . there
Prediction:	1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609
Target:	is root oppression is speakers speakers oppression ukraine is been been been been been is is



( the house rose and observed a minute 's silence ) .
Eisner Prediction:  root . . . . . . . . . . (
Prediction:	1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609 1609
Target:	rose house rose root observed rose minute silence minute observed rose rose