Thanks a lot for the insight @gphilip and @ptrblck . I have one last question. Ultimately, I want to verify that the partial derivatives for my functions after the last linear layer is being passed on correctly. Currently , I am trying to check  this feature by verifying the grads over the outputs of each functions. My question is–can I verify that all grads are working properly by checking the grad of the leaf tensor (the weights)? In other words, will the computational tree break down and not yield any grads for the leaf tensor if my intermediate functions are not passing on gradients, or will it simply calculate the gradients directly with respect to the loss function (disregarding the gradients of the intermediate functions)? Thank you very much.