Gautham_Kumaran:

I tried encoder-decoder model without attention like you have described and by passing my previous forecast as the input to the next decoder cell, but this didn’t give me an improvement over my previous model of feeding the results from all the encoder GRU cells to a Linear layer.


It is difficult to compare these…



 Gautham_Kumaran:

I can understand your point that using attention is resource-intensive, but provided I want to squeeze out the extra performance from the model, is this implementation of attention alright?


I haven’t noticed any coding/logic issues.



 Gautham_Kumaran:

Is there any other example of using attention for time-series forecasting?


I’m sure arxiv.org 16 has something on that