I am trying to replicate this:

  
      arxiv.org
  
  
    
1902.02443.pdf 9

1784.82 KB

  
  
    
    
  
  


Where essentially it says that the output of the embedding layer (figure 5) is Demb*Vocab_size.
I think because it runs through 30-40 batches without memory error tells me that there is enough memory to hold the weights, but it keeps increasing after every batch which signals some kind of memory leak.
By the way, where did you see that I first push the data through the LSTM layer?