Solved by enterthevoidf22 in post #13 
              
              
                ok! seems like the entire drama was solved by changing the Relu at the end of the network to LeakyRelu. 
i don’t fully understand what happened, maybe albanD can comment? 
my intuition is that the entire network is composed of LeakyRelus and norm layers, so it must contain a lot of negative activati…