He guys, I am using U-net and RNN.
I found my GPU memory is increasing after each step instead of remaining stable.
After about 40 steps, run out of memory error occurs.
Seems there is something stock in GPU memory and cause leak.
I am using RTX 2080 ti.
My code is here: https://github.com/vagr8/R_Unet/blob/master/R_Unet.py 2
Thanks!