It seems to be saying that you can always “differentiate” through if else conditionals because you only pass the gradient through the conditional that is selected… kind of like how max is differentiable because you only pass the gradient through the maximum value…
Are there any cases where program logic is not differentiable?