Solved by Peishi in post #3 
              
              
                Thank you for the reply! The problem is solved by simply feeding the GPU with each batch during training, instead of sending all data together. But the model parallel approach looks also interesting. Is there any pytorch tutorial for that?