Hello @XingxingZhang
similar to what you describe, my issue is with CPU RAM, too, GPU memory consumption is constant.
With the unidirectional LSTM, @apaszke 's hint works for me.
In more detail, you need to “import gc” at the top and call “gc.collect()” every epoch or so. (I don’t know whether it is always needed, but that worked for me.)
My impression was that the OpenNMT version in https://github.com/pytorch/examples 24 has some fixes that the OpenNMT does not have yet.
Best regards
Thomas