OcramDiGallura:

It’s actually both in different senses:
Memory usage is increasing on RAM, slightly, but increasing.
Momory usage is large and constant on GPU. It takes 5.7GB for the model and only one batch of data (10 sentences).


There is a bug in cudnn that makes rnns leaking CPU memory. It leaks very slowly. That is probably the cause of your first observation. From your second observation, it doesn’t seem a memory leak. There is probably some part of the graph that could be optimized. Could you try using a list of tensors for char_rep, char_hidden and hidden_state? Storing things in slices sometimes leads to copying the entire tensor multiple times.