I’m playing with rnn-char example from here: https://github.com/spro/practical-pytorch/blob/master/char-rnn-generation/char-rnn-generation.ipynb 3
Here’s the simplified code I’m running:

  
      gist.github.com
  
  
    https://gist.github.com/michaelklachko/fa485c27f83477981889d8a2fde0af4d 10
GRU based char-rnn model
import random
import torch
import torch.nn as nn
from torch.autograd import Variable

text = open('input.txt', 'r').read()
file_len = len(text)
all_chars = ''.join(set(text))
n_chars = len(all_chars)
        
This file has been truncated. show original




  
  
    
    
  
  


When I run it on GPU, memory usage is almost 12GB (Pascal Titan X). GPU utilization is ~20%.
I added ‘requires_grad=False’ to the input Variable, but that didn’t help.
Any ideas why so much memory is in use?
Is this normal for a RNN network of this size (3 layer GRU with hidden_size=500, and input_size=100?