Solved by Tyzcen in post #3 
              
              
                In case anybody ever stumbles upon a similar task. I found a way, which is pretty specific to what i need. 
Basically as the nets i am using are all fully connected sequential Networks, i can manually execute the forwards passes via batched matrix multiplications, which can take use of cudas paralle…