Solved by justusschock in post #4 
              
              
                This would not work good, since this required CPU and GPU to synchronize quite often(currently they are asynchronous) and synchronization slows down a lot. The negative effect of .to() for every intermediate output would probably be superior to the benefits of memory usage.