Solved by JuanFMontesinos in post #2 
              
              
                Well, if you have no real idea about why it works the best you can do is trying to plot the loss surface. There are some papers in which you can see that, for example, DenseNet loss has a single local minima and it’s a convex structure. Papers in the same line shows that residual connections makes l…