Your issue looks similar. The only element that could cause a memory leak is the outputs variable, if you are holding on to it. In my test, I avoid storing the result of net.call so that it can be freed immediately.
Also, I don’t encounter a memory problem when I use cuda.