https://discuss.pytorch.org/t/check-gradient-flow-in-network/15063/10 7
From here I tried using plot_grad_flow
to plot the gradient flow I was getting, and I get the following. Gradients does seem to vanish in the deeper parts of U-net.