Yeah you are right. It was not well explained sorry. So if I had somewhere a non differentiable op wouldn’t the gradients before that be all None since it is a continuous chain? In my case I get None only for the gradients of the bias from the last layer.