Hi,
I have created seq2seq model and it has encoder and decoder.
The encoder has 2 main “blocks”: embedding layer (nn.Embedding) and Lstm (nn.LSTM).
Lstm has a lot of parameters and GPU can’t handle even this one block so I can’t use lstm.cuda(1).
What should I do? How can I distribute memory through 8 GPUs in a case of RNN?
Thanks!
P.S Also, want to say that Pytorch is awesome, thank you for developing it