Ahmad_Nedal:

What I do not understand is that adding up the gradients (the link is above) gives zero in theory.


I am not sure to follow that. It gives 0 only in some cases no? In others it depends on the difference between the branches right?