Without attention, the context vector is usually just the last hidden state – the most basic encoder-decoder setup. And you’re already return the last hidden state in your forward() method of your encoder. Your current code is working, right?