Sorrry for the late reply! I thought I put notifications on. Yes you are right, I just ran it today for one 1 recursive depth(this trains 5 different networks of shapes (5 hidden layers,64 neurons) )This takes more than hour. Comparing it to the low resolution would be 12s.
I ran a test of the GPU Memory allocation and it seems that the problem is that I’m giving a lot of to the GPU and not erasing un necessary data.
You can see the spike of of memory allocation in this graph