Hi,
Problem summary: I have implemented a seq-to-seq encoder decoder model that uses Reinforcement Learning for model training. I have encountered following error while training (exactly when attention is being computed i.e.
intermediate = vector.matmul(self._w_matrix).unsqueeze(1) + matrix.matmul(self._u_matrix)):
‘’‘RuntimeError: CUDA out of memory. Tried to allocate 944.00 MiB (GPU 0; 11.17 GiB total capacity; 9.86 GiB already allocated; 310.81 MiB free; 10.58 GiB reserved in total by PyTorch)’’’
Model Details:

The encoder model encodes an input sequence using Pytorch’s LSTM cell (hidden_size=512).
The decoder model’s LSTM cell is initialised using encoder model’s output. And at every time-step(t), decoder model’s input is a concatenation of (a) previous time step’s prediction (t-1) and (b) attended input produced by attending to encoder outputs.
Training objective: We beam sample top-k predictions from the decoder model and generate the reward for each decoding. I am back-propagating loss = log probabilities of top-k decodings * their respective rewards (REINFORCE trick).

The attention mechanism to compute weights is the following:
Attention is computed between a vector, x (in our case h_t) and a matrix, y (outputs of encoder) using an additive attention function. The function has two matrices W, U and a vector V. The similarity between the vector x and the matrix y is computed as V tanh(Wx + Uy).
Parameters: I am sampling 64 top decodings, batch-size: 16, hidden size of 512 for every LSTM cell (encoder and decoder both), max sequence length of 50 when decoding using beam search.
I know this might be confusing without exact code. Let me know if any part of the code needs to be shared for more clarity. I want to debug what is consuming most of the memory because the same error appears on GPU with larger memory of 32GBs (tried on k80 and v100).