the decoder_outputs is already initialized, here is the completed code:

  
      github.com
  
  
    howardyclo/pytorch-seq2seq-example/blob/master/seq2seq.ipynb 7
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Batched Seq2Seq Example\n",
    "Based on the [`seq2seq-translation-batched.ipynb`](https://github.com/spro/practical-pytorch/blob/master/seq2seq-translation/seq2seq-translation-batched.ipynb) from *practical-pytorch*, but more extra features.\n",
    "\n",
    "### Extra features\n",
    "- Cleaner codebase\n",
    "- Very detailed comments for learners\n",
    "- Implement Pytorch native dataset and dataloader for batching\n",
    "- Correctly handle the hidden state from bidirectional encoder and past to the decoder as initial hidden state.\n",
    "- Fully batched attention mechanism computation (only implement `general attention` but it's sufficient). Note: The original code still uses for-loop to compute, which is very slow.\n",
    "- Support LSTM instead of only GRU\n",
    "- Shared embeddings (encoder's input embedding and decoder's input embedding)\n",
    "- Pretrained Glove embedding\n",
    "- Fixed embedding\n",
    "- Tie embeddings (decoder's input embedding and decoder's output embedding)\n",


  This file has been truncated. show original

  
  
    
    
  
  


I independently test the decoder’s forward iteration with some fixed data, sometimes it add memory per timestep, but not, which is weird.