Hello,
I have a dataset of questions and answers. I have implemented GRU with seq2seq network using pytorch. I manage to good predictions but I wanted better so I implemented attention. I have much better predictions but the loss is bigger I used the same parameters(hidden, batch, epochs)
So my questions are:
How can this happen?
Should I use the same parameters in order to compare?
Bellow you can see my graphs:
Without Attention:

The loss is:  [35.00632381439209, 15.949099481105804, 11.76700609922409, 8.921187818050385, 5.385168731212616, 4.149292141199112, 3.3940756022930145, 2.9950645118951797, 3.0654407143592834, 2.6672486662864685, 2.3585961014032364, 2.177135296165943, 1.6265693753957748, 1.5573233589529991, 1.3611307963728905, 1.2601154148578644, 1.159318681806326, 0.9464887417852879, 0.9394438974559307, 0.8580565229058266]
With Attention:
The loss is:  [35.52947795391083, 25.49850583076477, 19.276048064231873, 14.980394184589386, 12.132617473602295, 10.224796116352081, 7.703132122755051, 5.529960989952087, 4.445598945021629, 3.951431155204773, 3.587742894887924, 3.5412986427545547, 3.1525653302669525, 3.1754210591316223, 3.0430017858743668, 3.0690969228744507, 3.1617940813302994, 2.813789874315262, 3.0119248926639557, 2.8593120127916336]
Thank you very much in advance!
Kind regards,
Angelos